Vision Targets

A vision Target describes your page as settings: where it is, where its browser runs, and the secrets the Driver types. A Target is the page and holds no steps. The Driver's prompt holds every one, the sign-in and the way to the chat included, and Okareo makes each one a turn of the conversation.

Okareo UI

1. Go to Targets and click New Target.

2. Switch the type selector to Vision, and fill in the fields in the order the form shows them:

   • Name: the name Okareo lists the Target under, such as Portal.
   • URL: the page every conversation opens, an http or https address, such as https://portal.example.com/login.
   • Sign-out URL (optional): an address Okareo opens when a conversation ends, your site's sign-out, such as https://portal.example.com/logout.
   • Max Parallel Requests (optional): how many conversations a Run holds open at once, such as 5. Leave it empty for the default, 10.
   • Secrets (optional): one row for each value the Driver types, added with Add secret: its name and its value. This page has one, PORTAL_PASS. See Secrets.

3. Click Create. The Target shows under Targets with the type Vision, and its page reads the settings back, the secrets by name only. Pick it on the simulation page with a Driver and a Scenario, as in Running from the UI.

Python

from okareo_vision import BrowserConfig, VisionTarget

portal = VisionTarget(
    url="https://portal.example.com/login",
    sign_out_url="https://portal.example.com/logout",
    browser_config=BrowserConfig(host="hosted", max_parallel=5),
    secrets=["PORTAL_PASS"],
)

Pass the Target to run_simulation as Target(name="Portal", target=portal), as in Your First Vision Simulation. The name in Target(name=...) is the one Okareo lists.

The Driver for this page opens its prompt with the steps to the chat. {scenario_input.email} comes from the row, and {secret.PORTAL_PASS} is typed by name:

## Getting to the chat

Fill Email with {scenario_input.email} and Password with {secret.PORTAL_PASS}, then press Log In.
When the page shows Your Account, press Chat with us.
Your first message goes in the box named Type your message; until it shows, press or wait instead.

Settings

Setting	Default	What it does
url	required	The page every conversation opens, an http or https address. {scenario_input.KEY} in it is filled from the row's input, per conversation.
browser_config	required: its host	The browser and where it runs, as a BrowserConfig; see BrowserConfig.
sign_out_url	None	An address Okareo opens when a conversation ends, before it closes the page: your site's sign-out, so a session does not outlive its conversation. It is not a turn, and if it fails the page closes anyway.
settle_timeout_s	30.0	How long a turn may wait for the agent's reply, in seconds. At most 240.
secrets	None	The secrets the Driver's prompt types, by name; see Secrets.
fire_events	()	Events fired on the page after it loads, in order; see Page Events.

Every conversation starts with no cookies or sign-in left from another. On a page with a sign-in, the Driver signs in every conversation.

The Target form in the Okareo app sets url, sign_out_url and secrets, and browser_config.max_parallel as Max Parallel Requests. Every other setting is set in Python, and an edit in the app keeps it as it is.

Where the Browser Runs

BrowserConfig(host=...) says where the browser runs, and every VisionTarget must name it. Okareo drives the browser either way.

	host="hosted"	host="client_relay"
The browser	A browser Okareo hosts, a new one for each conversation.	The Chrome on your machine, driven by Okareo through the relay.
The page	One the internet can reach.	Any page your machine can reach, such as one on your own network or on your laptop.
On your machine	okareo-vision.	okareo-vision, okareo 0.0.163 or newer, and Google Chrome or Chromium.
The secrets' values	Sent to Okareo when the Target is registered. Okareo keeps them for its browser to type, and leaves them out of every API answer.	Never leave your machine. The relay types each one where the Driver names it, and Okareo gets the names only.
Your process	Registers the Target and waits for the Run.	Registers the Target, waits for the Run, and runs the relay and Chrome until the Run finishes.
In the Okareo app	Made with the Target form, and run from the simulation page.	Registered and run from Python only. Its Runs show in the app like any other.

A third host, "local", is for Okareo's own development; use "hosted" or "client_relay".

Whichever host you pick, the process that starts the Run reaches Okareo over HTTPS and over a WebSocket to wss://connect.ngs.global:443. A firewall must let both out.

On the Relay

The relay runs inside the process that calls run_simulation. Chrome opens when the Run's first conversation starts, and closes when the Run ends.

• Which Chrome. The relay starts the Google Chrome or Chromium installed on your machine. To use another, set OKAREO_RELAY_CHROME to its path.
• Watching it. Chrome runs without a window by default. BrowserConfig(host="client_relay", headless=False) shows a window for each conversation. A window needs a display; on a Linux server, run under xvfb-run.
• A proxy. Chrome reaches the page directly from your machine, and does not read HTTPS_PROXY. A page behind a proxy needs it as a flag: browser_args=("--proxy-server=http://proxy.example.com:8080",).

Secrets

A secret is a value the Driver types without seeing it: a password, a fixed code, an authenticator's six digits. The Target lists each one by name, and the Driver's prompt names it as {secret.NAME}, as the whole value of a field:

Fill Password with {secret.PORTAL_PASS}, then press Log In.

Okareo types the value into the field, and keeps it out of what the Driver, the app and the Checks see.

Okareo UI

In the Target form, each secret is a row under Secrets: its name and its value. Add secret adds a row, and the trash icon removes one. Okareo types the value as you gave it, such as a password or a fixed code. An authenticator's six digits are set up in Python, with Authenticator(seed); an edit in the app keeps such a secret an authenticator's.

Okareo holds each value you type. Once the Target is saved, the form shows each value as Secret redacted, and the Target's page lists the secrets by name only. To change a saved secret, click Replace, confirm, and type the new value; its name can change then too. A secret you leave as it is keeps its saved value.

Python

Give the secrets as names or as values:

import os

from okareo_vision import Authenticator, BrowserConfig, VisionTarget

# Names: each value is read from the environment variable of that name.
portal = VisionTarget(
    url="https://portal.example.com/login",
    browser_config=BrowserConfig(host="hosted"),
    secrets=["PORTAL_USER", "PORTAL_PASS"],
)

# Values: a string is typed as it is; Authenticator(seed) types the current six-digit code.
portal = VisionTarget(
    url="https://portal.example.com/login",
    browser_config=BrowserConfig(host="hosted"),
    secrets={
        "PORTAL_PASS": os.environ["PORTAL_PASS"],
        "PORTAL_CODE": Authenticator(os.environ["PORTAL_TOTP_SEED"]),
    },
)

unusable_secrets() names the secrets that cannot be typed now (an environment variable that is not set, or a seed that is not one), so a script can stop before it starts a Run:

unusable = portal.unusable_secrets()
if unusable:
    raise SystemExit(f"Set these before the Run: {', '.join(unusable)}")

• A name is letters, digits and _, and does not start with a digit.
• An authenticator's seed is the text under the QR code an authenticator app scans. Okareo types the six digits the app would show at that moment.
• On a browser Okareo hosts, the values go to Okareo when the Target is registered, a seed included. A secret your machine holds no value for is refused there, by name.
• On the relay, the values stay on your machine. The relay reads each one when it types it.
• A secret the Target does not list cannot be typed: a Driver's prompt that names one is refused when the Run starts. Add it to secrets, or take it out of the prompt. On the simulation page, the Target's card names any such secret once the Target and the Driver are picked, before the Run.

Never write a secret's value in the Driver's prompt, a Scenario row, or a Check.

Page Events

Some pages show their chat, or switch a feature on, when an event fires on the page. fire_events fires those events after the page loads, in order, as a CustomEvent on the page's window with the name and detail you give:

from okareo_vision import BrowserConfig, PageEvent, VisionTarget

store = VisionTarget(
    url="https://shop.example.com/",
    browser_config=BrowserConfig(host="hosted"),
    fire_events=[PageEvent("show-chat", detail=True)],
)

A page event is data, a name and a detail. Nothing you give is run as script.

BrowserConfig

BrowserConfig is the browser's settings as one group. Only host is required; a field left out keeps its default.

Field	Default	What it does
host	required	"hosted" or "client_relay"; see Where the Browser Runs.
viewport	(1280, 900)	The size each conversation's page opens at, as width and height in pixels.
max_parallel	10	How many conversations a Run holds open at once. On the relay, this is also how many pages the Chrome on your machine holds open at once.
open_spacing_s	0.0	Seconds between the starts of a Run's conversations, so their pages open that far apart. 0 starts them all at once.
page_open_timeout_s	30.0	How long a page may take to load, in seconds. At most 240.
headless	True	On the relay only. False shows a window for each conversation.
browser_args	()	On the relay only. Extra flags Chrome starts with, such as --proxy-server=... or --host-resolver-rules=....

BrowserConfig also takes channel and user_agent. No browser reads them today, so leave them unset.

Upgrading from 0.3

In 0.4, okareo-vision no longer drives a browser itself. Okareo does, and a Target is settings only.

• Move the steps to the Driver. Whatever your Target's code did on the page (close a notice, sign in, open the chat) goes at the top of the Driver's prompt, in plain words, one screen at a time.
• Name where the browser runs. Add browser_config=BrowserConfig(host="hosted"), or host="client_relay" for a page only your machine can reach. A Target with no host is refused.
• Register the Target again. A Target saved by 0.3 is refused when a Run starts with it. Running run_simulation with the new VisionTarget registers it again.
• Sign in every conversation. No sign-in is saved between conversations. The Driver signs in, with its passwords and codes as secrets.
• Raise max_turns. The steps to the chat are turns now, and count toward it.
• Start Runs with run_simulation. The okareo-vision command is gone.
• Install no browser for a hosted Target. A Target on a browser Okareo hosts needs no browser on your machine. On the relay, Okareo uses the Chrome or Chromium installed there.

Common Problems

The Page Never Shows a Chat

Open the conversation in the Run's page in Okareo. It shows each of the Driver's presses and the screen after it, so the step where the page did something the prompt did not expect is there to read. The Driver's prompt is where the chat gets opened: its first steps name the controls to press, and say that the first message goes in the message box once it shows.

A Conversation Runs Out of Turns Before the Chat

The sign-in and the steps to the chat are turns, and count toward the Run's max_turns. Raise it.

The Site Refuses Headless Chrome

Run the Target on the relay with headless=False. Sites with a bot check commonly let in only a real Chrome with a window. On a Linux server, run under xvfb-run.

The Relay Cannot Start Chrome

Install Google Chrome or Chromium, or set OKAREO_RELAY_CHROME to the path of the Chrome to start.

A Relay Target Is Refused When It Is Built

A Target on host="client_relay" needs okareo 0.0.163 or newer. Upgrade it with pip install -U okareo.

Sign-In Needs MFA

A code your machine can produce works: a test account's fixed code as a plain secret, or an authenticator app's seed as Authenticator(seed). A sign-in only a person can pass, such as a push to approve or a code sent to a phone, cannot run unattended.

The Run Will Not Start Because of a Secret

The Driver's prompt names a secret the Target does not list: add it to secrets, or take it out of the prompt. On a browser Okareo hosts, a secret your machine holds no value for is refused when the Target is registered: set its environment variable, and check with unusable_secrets().