API service used to generate rules for Consent-O-Matic chrome extension

Setup

Install Poetry on your system. You can do this with this command:
curl -sSL https://install.python-poetry.org | python3 -
Activate the virtual environment:
poetry shell
Install all dependencies:
poetry install
Add Python to path:
export PYTHONPATH=${PYTHONPATH}:$(pwd)

If you don't want to use docker and pass your OpenAI's key there, you can run this without it, just export your API key:
export OPENAI_API_KEY='your_api_key'

Development

Start the server with hot reloading:

poetry run uvicorn server.main:app --reload

Lint your code by running:

poetry run pre-commit run -a

Code examples

Browser emulator

def test_browser_emulator():
    be = BrowserEmulator()
    be.driver.get("https://harrysdolcevita.com/")
    HTML_before = be.check_parent_html_by_aria_label("Reject All")
    be.click_element("xpath", be.create_area_label_xpath("Reject All"))
    HTML_after = be.check_parent_html_by_aria_label("Reject All")
    assert HTML_before != HTML_after

Testing

Run all tests:

poetry run pytest

Add additional tests in tests directory. The test file should follow such format: test_*.pywhere * would be substituted with the name of the test.

Docker usage

docker build -t rulegen-api:1.0.0 .

docker run -p 8000:8000 -e OPENAI_API_KEY='your_api_key' rulegen-api:1.0.0

API endpoints

To view all API endpoints, visit http://localhost:8000/docs

Rule Generation (<tt>rulegen/generator.py</tt> Overview)

The rulegen/generator.py module is responsible for the actual extraction of the rules from provided HTML. It utilizes models from rulegen/models.py and integrates with OpenAI's GPT-3.5 for dynamic rule creation. Below are its main functionalities:

RulegenClient Class

Client Initialization: Leverages the instructor library to patch an OpenAI client for interfacing with GPT-3.5.
System Tasks: Defines various system tasks such as finding buttons to reject cookie policies, open settings, and confirm choices.
Model Extraction and JSON Saving:
- save_model_as_json: Saves the generated model in JSON format.
- extract_model: Extracts the model based on a system task and user content, and generates responses using the specified response model.
Selector Retrieval:
- get_selector: Retrieves CSS selectors based on content, useful for identifying specific elements in a web application.

Rulegen Models Overview

The rulegen/models.py file contains several Pydantic modeled after consent-o-matic rule definitions. These models define the structure for various elements like style filters, DOM selections, matchers, actions, detectors, methods, and the main CMP (Consent Management Platform) rules. Below is a brief overview of each model:

StyleFilter

Defines CSS style property filters. Includes fields for the CSS property option, value, and a negation flag.

DOMSelection

Represents a DOM selection with various filters like CSS selector, text filter, style filter, display filter, iframe filter, and child filter.

CssMatcher & CheckboxMatcher

Define matchers for CSS and checkbox elements in the DOM. These are used to identify specific elements on a web page.

BaseAction & ClickAction

Represent actions that can be performed on DOM elements. ClickAction is a specific type of action for simulating mouse clicks.

MatcherType

A union of CssMatcher and CheckboxMatcher, used for defining present and showing matchers in the Detector model.

Detector

Used to define detection criteria for the Consent Management Platform, including present and showing matchers.

Method

Defines methods with optional actions and names. The NAME literal defines specific method names like OPEN_OPTIONS, DO_CONSENT, etc.

CMP

Consent Management Platform model that combines detectors and methods for rule definition.

Rules

The main model encapsulating the CMP rules and schema information for the Cookie Popup Manager.

These models are central to the operation of the Cookie Popup Manager, providing a structured approach to defining and managing consent rules.