Apps Use Click to Do in Windows 11

This tutorial will show you how to use Click to Do to perform AI actions with text and images for your account on a Windows 11 Copilot+ PC.

Click to Do (preview) helps you get things done faster by identifying text and images on your screen that you can take actions with. Click to Do analyzes what's on screen and then you can choose the text or image you want to take action on.

The analysis of this screenshot is always performed locally on your device. Analysis only begins after you actively engage with Click to Do and ends when you exit Click to Do. Click to Do only identifies text and images, not the content of those text or images. It does not analyze any content in, for example, minimized applications that are not on your screen.

Content is only shared if you choose to complete an action like Search the web. You’ll notice that when Click to Do is active, your cursor is blue and white. The cursor also changes shape depending on the type of info beneath it. What you can do with the info changes based on what kind of content Click to Do detects.

Starting with Windows 11 build 26100.4202 (24H2), Ask Copilot is a new action within Click to Do. When you highlight either text or image with Click to Do, you’re given an option to Ask Copilot. Microsoft Copilot opens with your content in the prompt box. You can send selected text or image within Click to Do Directly to Copilot app. The selected text or image is handed off to the Copilot app where you can complete your prompt. This feature is not available in the European Economic Area (EEA).

Starting with Windows 11 build 26100.3915 (24H2), Click to Do (preview) makes it easier than ever for you to take immediate action on whatever catches your eye on-screen on your Copilot+ PC, saving you time by helping complete tasks inline and quickly getting you to the app that can best complete the job. For example, select an image using Click to Do to take actions like erasing objects using the Photos app or remove the background using Paint. To use Click to Do, simply use Windows key + mouse click or Windows key + Q. You can also enter through the Snipping Tool menu and print screen or searching “Click to Do” in the search box on the Windows taskbar. On Snapdragon-powered Copilot+ PCs, Click to Do includes intelligent text actions. Use Windows key + mouse click or Windows key + Q to select a text block and then drag to select the text that you want. You’ll see options to Summarize or to help you Rewrite your text, so it sounds more causal or more formal. These text actions leverage the capabilities of Phi Silica, the on-device Small Language Model (SLM) that is built right into Windows.

Starting with Windows 11 build 26100.4202 (24H2), Microsoft is introducing more intelligent text actions to enhance your Click to Do experience even further on AMD and Intel™-powered Copilot+ PCs. Use WIN key + mouse-click or WIN + Q to select a text block and then drag to select the text that you want. You’ll see options to Summarize, Create a bulleted list, or to help you Rewrite your text, so it sounds more causal, more formal, or more polished.

Starting with Windows 11 build 26100.4484 (24H2), Windows 11 users with a Copilot+ PC using Click to Do can now use a new action, Ask Microsoft 365 Copilot, to send text or images to Microsoft 365 Copilot and have it address your queries. The Ask Microsoft 365 Copilot action will require a Microsoft 365 license and respect your organization’s privacy policies. Click to Do is adding a new capability that enables you to take actions from an email address. These new actions make light work of connecting with a coworker or friend through Microsoft Teams. Have a question that you need an answer to? With just a couple clicks, you can easily send a Teams message. If you need to really talk something out, you can find time with your contacts and schedule a meeting. These new skills are provided when selecting any email address recognized by Click to Do on your screen, where you can then choose the “Schedule a meeting with Teams” or “Send a message with Teams” action and stay in the flow. Microsoft Teams version 25153.1002.3699.5695 and higher is required.

Starting with Windows 11 build 26120.4733 (Beta 24H2) and build 26200.5702 (Dev 25H2), Microsoft is introducing a new “describe image” action in Click to Do to get detailed descriptions of images, charts and graphs – useful to get a quick overview of the visual content. When you use the action for the first time, the required models are set up, and the descriptions are generated locally on your device making sure your sensitive data stays on your PC. The new describe image action in Click to Do is rolling out to Windows Insiders on Snapdragon-powered Copilot+ PCs, with support for AMD and Intel-powered Copilot+ PCs coming soon.

Starting with Windows 11 build 26120.4741 (Beta 24H2) and build 26200.5710 (Dev 25H2), new describe image action in Click to Do rolling out for AMD and Intel powered Copilot+ PCs.

Starting with Windows 11 build 26100.4770 (24H2):

• New! 1 Practice in Reading Coach is a new Click to Do text action that helps you improve reading fluency and pronunciation. Select text on your screen, choose Practice in Reading Coach, and read the text aloud. Reading Coach gives you feedback and shows where to improve. To use this feature, install the free Microsoft Reading Coach app from the Microsoft Store.
• New! 1 Read with Immersive Reader is a new text action in Click to Do that displays text in a focused, distraction-free environment. It helps improve reading and writing for all skill levels and abilities. You can adjust text size, spacing, font, and background theme, have text read aloud, break words into syllables, and highlight parts of speech. The picture dictionary shows images for unfamiliar words. To use this feature, install the free Microsoft Reading Coach application from the Microsoft Store.
• New! 1 With the Draft with Copilot in Word text action, you can quickly turn any recognized text into a full draft. Whether it’s a sentence in an email or a snippet on your screen, press Win + Click on the recognized text, then select Draft with Copilot in Word. No more blank pages. No more writer’s block. Just momentum. To use “Draft with Copilot in Word” a Microsoft 365 Copilot subscription is required.
• New! 1 Click to Do on Copilot+ PCs now supports actions through Microsoft Teams. When you select an email address recognized by Click to Do on your screen, you can choose to send a Teams message or schedule a Teams meeting. These options make it easy to ask a question or set up time to talk without interrupting your workflow.

Starting with Windows 11 build 26120.5722 (Beta 24H2) and build 26200.5722 (Dev 25H2), Microsoft is beginning to roll out a new tutorial for Click to Do. When you launch Click to Do for the first time, you’ll be offered a quick interactive tutorial that shows you how to accomplish tasks faster. During the tutorial, you’ll take actions on both text and images, so you’ll know how to quickly get a summary of a large block of text or remove the background from an image. You can revisit the tutorial at any time by selecting …, then Start tutorial.

Starting with Windows 11 build 26100.7019 (24H2) and build 26200.7019 (25H2), Microsoft is adding these new features to Click To Do:

• New! The prompt box in Click to Do streamlines interaction with Copilot, helping you work more efficiently. You can type a custom prompt directly into the text box, which sends your prompt and selected on-screen content to Copilot. Suggested prompts appear below the text box and are powered locally by Get started with Phi Silica in the Windows App SDK. These suggestions are available for text selections in English, Spanish, and French. This feature isn’t currently available in the European Economic Area (EEA) or China.
• New! You can now translate on-screen text with Microsoft Copilot with Click to Do. When you select text that is in a different language from your Windows display language or preferred language setting, a translation suggestion appears. After selecting the option to translate the text, the translated text will appear in the Copilot app. You can see the supported regions and languages in Microsoft Copilot. This experience is not yet available for customers in the EEA (European Economic Area) or China.
• New! Click to Do provides unit conversions for length, area, volume, height, temperature, and speed. When you hover over a number + unit, a floating tooltip will show you the conversation. When you select the number + unit, the context menu opens with additional conversation options. You can also access more conversions via the Copilot app. This experience is not yet available for customers in the EEA (European Economic Area) or China.
• New! By pressing and holding two fingers anywhere on your Copilot+ PC with a touch screen, you can simultaneously launch Click to Do, select the entity under your finger and see relevant actions.
• New! You can now select objects in Click to Do using Freeform Selection, Rectangle Selection and Ctrl + Click. To use Freeform Selection Mode, select the Freeform button on the toolbar. Then use your pen or finger to draw freely around the items you want to select. To use the Rectangle Selection Mode, select the Rectangle Selection button on the toolbar. Then drag a box around the items. Everything inside the rectangle will be selected. You can also use Ctrl + Click to select multiple types of items—such as text and images—by holding down the Ctrl key and selecting each item.
• New! Click to Do can now detect tables. You can now highlight any simple table and send it to Excel, copy or share it. With a table in any application, you can press Win + Click to invoke Click to Do, or Win + Q or right swipe and tap to select the table. Once selected, you’ll see the actions you can take like Convert to table with Excel. Just click, capture and continue. This is available on Snapdragon-powered Copilot+ PCs, with support for AMD and Intel-powered Copilot+ PCs coming soon. You’ll need the latest Microsoft Excel application installed on your PC to see Convert to table with Excel and a Microsoft 365 Subscription. This change is not yet rolling out to Windows Insiders in the EEA (European Economic Area).
• New! Live Persona Cards from Microsoft 365 now appears in Click to Do. To view a profile card, press Win + Click on an email address from your organization.
• New! Visual cues now make key items, such as emails, tables, and more, light up on your screen as you launch Click to Do.
• Fixed: Click to Do may unexpectedly invoke sometimes when pressing Windows key + P.

Starting with Windows 11 build 26100.7309 (24H2) and build 26200.7309 (25H2), the Click to Do context menu is being updated with a streamlined design, making it simpler to locate what you need. Frequently used actions like Copy, Save, Share, and Open will now be easier to access directly from the context menu. Whenever a large image or table appears on your screen, the context menu will automatically pop up, making it quicker and easier to access the actions and results you need. Microsoft is temporarily turning off Image Object select as detailed here for Dev and Beta.

Starting with Windows 11 build 26220.7271 (Dev and Beta 25H2), Microsoft is testing and refining the Click-to-Do top bar to determine the best experience for future updates. Functionality will vary by device and market.

Starting with Windows 11 build 26220.7523 (Dev and Beta), the Copilot prompt box in Click to Do now supports Microsoft 365 Copilot. Ask Microsoft 365 Copilot about what’s on your screen. Quickly type a custom prompt directly into a text box, which is then sent to Microsoft 365 Copilot along with your selected screen content. This change is not yet rolling out to Windows Insiders in the EEA (European Economic Area) or China. This feature previously rolled out last September to Dev and Beta channel outside of EEA.

References:



Here's How:

1 Open Click to Do when there's something on your screen you want to use Click to Do on.

2 You can perform any of the following actions below for text, images, URLs (site links), and email addresses found on your screen by Click to Do:



A) Type the text you want search for on the screen in the Find on screen box on the toolbar at the top. (see screenshot below)

B) You can click/tap on the search up and down caret symbols to cycle through the text found on your screen.

C) You can click/tap on X (close) to stop and reset search.





A) Click/tap on text you want to have the following available options: (see screenshots below)

• Copy: Copies the selected text to the clipboard
• Open with: Opens the selected text in a text editor like Notepad
• Search the web: Does a search for the selected text using Microsoft Edge and Bing

When the highlighted text is an email address, you can either copy the text or Send email.

• Send email: Uses your default email app to open a new email to the selected email address

When the highlighted text is a website, you can either copy the text or Open website.

• Open website: Opens the selected website in your default browser

More intelligent text actions are available when your default language is set to English, at least 10 words are selected, and you’re signed in with either a Microsoft account or Microsoft Entra account. Intelligent text actions, powered by the Phi Silica small language model and the NPU, include:

• Summarize: Briefly outlines the main points of the selected text
• Create a bulleted list: Creates a bulleted list of key details from the selected text
• Rewrite (Casual): Revises the selected text using a casual tone
• Rewrite (Formal): Revises the selected text using a formal tone
• Rewrite (Refine): Revises the selected text, making grammatical improvements

A) Click/tap on an image you want to have the following available options: (see screenshot below)

• Ask Copilot: Opens Copilot and uploads the image to ask Copilot about.
• Describe image: Get detailed descriptions of images, charts and graphs – useful to get a quick overview of the visual content.
• Visual search with Bing: Does a visual search using Bing using your default web browser
• Blur background with Photos: Blur the background with the Photos app
• Erase objects with Photos: Erase unwanted objects in the image with the Photos app
• Remove background with Paint: Remove the background of the image with Paint
• Copy: Copies the image to the clipboard
• Save as: Save the image to a specified location
• Share: Opens the share files option for the image file
• Open with: Open the image with an app such as Photos, Snipping Tool, or Paint.

That's it,
Shawn Brink