Creating Vision Activities with the Vision Assistant

What are Vision Activities with the Vision Assistant?

A Vision activity checks a learner's work by looking at their screen. When a learner selects Check or Score, Skillable Studio captures a screenshot of the virtual machine and uses AI to decide whether the screen matches the outcome you described.

The Vision Assistant is an AI chat built into the Vision activity editor. Instead of writing a prompt from scratch, you describe what the learner should have done, optionally attach a screenshot, and the assistant drafts a structured validation prompt for you. You review it, refine it in conversation, and confirm it before you save.

Upgrading

Currently we are working on an upgrade path from the previous version of Vision to the new chat based version of Vision. Today the only way to update a Vision activity to the new version is to delete and recreate the activity.

Why it matters

Writing a reliable validation prompt takes practice. Vague prompts produce inconsistent results, and reference images can drift out of date as software interfaces change. The Vision Assistant helps you:

• Write stronger prompts without prompt-engineering expertise. The assistant turns your plain-language description into a structured prompt with a clear summary and expected result.
• Ground the prompt in what's really on screen. Attach a live screenshot from your running lab so the assistant can see exactly what you're validating.
• Score more accurately. Evaluation uses your confirmed prompt and the learner's live screen only. No stored reference image is involved, which removes a common cause of incorrect results.
• Give learners useful feedback. When a check fails, learners can see an explanation of what the AI observed and what to correct - optional.

Who can use this

Anyone who can edit lab instructions in Skillable Studio can create and edit Vision activities providing their organization has opted-in to the Skillable AI Features. Vision activities are only available in labs that contain a virtualized environment.

Create a Vision activity with the Vision Assistant

1. Open your lab profile and click Edit Instructions.
   Tip: To attach live screenshots, open the instructions editor from a running lab instance. You can still chat with the assistant without a running lab, but Take a Screenshot is unavailable.

2. Select the Activities tab, then click + New Vision Activity.
3. In Choose Environment, select the virtual machine the AI should evaluate.
4. In Configure Prompt with the Vision Assistant, work with the Vision Assistant to generate a prompt.
5. Click Confirm Prompt once you're happy with the prompt shown in Prompt Output.
6. Click Next and configure the activity settings.
7. Click Save or Save and Insert.

Work with the Vision Assistant

The prompt step has two panels side by side: the Vision Assistant chat on the left and Prompt Output on the right. Drag the divider between them to resize either panel. To work in a larger window, click the expand icon in the Vision Assistant header.

Describe the outcome

In the chat box, which shows What would you like to create for this Vision Activity?, describe what the learner's screen should show when they've completed the task. Be specific about the outcome, not the steps. For example:

The learner has created a storage account named "labstorage01" in the East US region, and the Overview page for that account is open.

The assistant may ask follow-up questions. Answer them in the chat to sharpen the prompt.

Add visual context

You can give the assistant an image to work from. Images help it understand the exact screen, layout, and values you expect.

• Take a Screenshot: Captures the current screen of the virtual machine selected in Choose Environment and attaches it to your next chat message. Your lab must be running. Set the virtual machine to the expected end state before you capture.
• Upload an image: Use the attach button in the chat to upload an image from your computer.

Images are optional. They give the assistant context while you write the prompt. They aren't saved with the activity, and they aren't used when learners are scored.

Review the generated prompt

When the assistant has drafted a prompt, it appears in Vision Assistant chat window as a series on statements and rationale written in JSON. If one or more statements are incorrect or missing continue using chat to get the correct statement set. Once the prompt meets the requirements, press the Confirm Promptbutton.

Confirm the prompt

When the Confirm Prompt button has been pressed this presents the prompt in easy to read English for review. The output is presented in two parts:

If the prompt isn't right, keep chatting. Ask the assistant to tighten wording, add a condition, or remove something it got wrong.

Start over with Reset

Click Reset to clear your work. Depending on what's there, you can choose:

• Reset Chat History: Clears the conversation with the Vision Assistant and keeps the current prompt.
• Reset Prompt: Clears the prompt in Prompt Output and keeps the conversation.
• Reset Both: Clears both and starts fresh.

If you're editing an activity that's already saved, resetting doesn't change the saved activity until you click Save again.

Configure the activity settings

After you confirm the prompt, click Next to configure the rest of the activity. These settings work the same way as for other Vision activities:

• Name: The activity name. Visible to lab authors in the Activities tab, hidden from learners.
• Replacement Token Alias: A recognizable name for the activity token in your lab's Markdown. A default is generated for you.
• Skills (optional): Tag the activity with skills from your organization's skills framework, if one is configured.
• Instructions (optional): Text shown in the Instructions panel above the Check or Score button.
• Evaluation: Vision activities use On-demand evaluation, so learners click a button to have their work checked. You can also set Custom evaluation button text and Allow retries, with an optional Maximum attempts.
• Scoring: Select Scored and set a Score Value if the activity counts toward the learner's score.
• Settings: Show results in reports, Required for submission, and Blocks page navigation until answered.
• Feedback for User: Set Correct Answer Feedback and Incorrect Answer Feedback. Keep Show AI Feedback for Incorrect Answers selected to show learners the AI's explanation when a check fails.
• Outcomes: Click Add Outcome to branch the lab experience based on the result. See Activity Outcomes.

Edit a Vision activity

1. Go to Lab Profile > Edit Instructions > Activities.
2. Find the activity and click Edit.

When you edit an activity created with the Vision Assistant, the assistant opens already aware of your saved prompt, and the chat shows How would you like to improve this Vision Activity? Your saved Summary and Expected Result appear in Prompt Output. Ask the assistant for changes, confirm the updated prompt, and click Save.

Vision activities created before the Vision Assistant was available still open in the original prompt editor and keep working as they did before.

What learners experience

Nothing changes for learners. When a learner selects Check or Score:

1. Skillable Studio captures a screenshot of the virtual machine you chose in Choose Environment.
2. The AI compares that screenshot against your confirmed prompt.
3. The learner sees Correct or Incorrect, or your custom feedback text.
4. If the result is incorrect and Show AI Feedback for Incorrect Answers is selected, the learner also sees an explanation of what the AI observed and what to correct.

Common scenarios

Validating a cloud portal configuration. You're building a lab where learners configure a resource in a web portal that has no convenient API to script against. You launch the lab, complete the task yourself, and click Take a Screenshot on the finished configuration page. You tell the assistant which values matter: the resource name, region, and tier. Then you confirm the prompt it generates. Learners are scored on the outcome visible on screen, not on how they got there.

Tightening a prompt that's too strict. A pilot learner fails a check even though their work is correct, because the prompt expected an exact window layout. You edit the activity and ask the assistant to focus on the configured values and ignore window position and theme. You confirm the revised prompt and save.

Starting over after a wrong turn. Partway through a conversation, you realize you described the wrong end state. Click Reset, choose Reset Both, and describe the correct outcome from scratch.

Tips and best practices

• Describe outcomes, not steps. "The firewall rule allowing port 443 is listed and enabled" works better than "The learner opened the firewall settings and added a rule."
• Name the values that matter. Specific names, numbers, and states give the AI something concrete to check.
• Capture the finished state. When you take a screenshot, make sure the virtual machine shows exactly what a successful learner would see.
• Choose the right virtual machine first. Take a Screenshot and learner evaluation both use the machine selected in Choose Environment.
• Test as a learner. Before you publish, launch the lab, complete the task, and click Check to confirm the activity behaves as you expect. Then try it with the task left incomplete.

For more guidance, see Effective AI prompting.

If something doesn't work

Take a Screenshot is unavailable.
Your lab environment must be running to take a screenshot. Launch the lab and open Edit Instructions from the running instance. You can still upload an image through the chat attach button.

Confirm Prompt is unavailable.
The assistant hasn't produced a complete prompt yet. Keep the conversation going, or ask the assistant directly to generate the prompt.

"Vision Assistant returned malformed prompt data. Please try again."
The assistant's response couldn't be read as a prompt. Send your request again, or rephrase it.

"Please choose a target virtual machine."
Select a virtual machine in Choose Environment before you save.

The activity won't save.
Make sure you've clicked Confirm Prompt, entered a Name and Replacement Token Alias, and set a Score Value if the activity is scored.

Reset is unavailable.
There's nothing to reset yet. Reset becomes available once you've sent a message or there's a prompt in Prompt Output.

Limitations

• Vision activities are only available in labs with a virtualized environment.
• Evaluation is on-demand only. Learners must click Check or Score.
• Images you attach in the chat are used as authoring context only. They aren't saved with the activity, so attach a new one if you need visual context in a later editing session.
• Each activity evaluates one virtual machine, the one selected in Choose Environment.
• Vision activities created before the Vision Assistant open in the original editor and can't be edited with the Vision Assistant.