The latest version of the iPhone operating system, iOS 15, has several new features that use artificial intelligence to analyze the content in a photo.
While object recognition has been around for a while, Apple says that its implementation differs because it happens on the device, instead of on a cloud server. Plus, it’s built into the operating system.
One of the handiest applications of this new technology is the ability to copy and paste text from a photo — so, for example, users can take a photo of a page in a book and then grab that text and put it into a document or a note.
It also allows users to call or email more easily by recognizing phone numbers and email addresses that have been written down, like on a business card.
It works with handwritten text as well — Apple’s example when it announced the feature in June was turning scribbles on a whiteboard into notes.
Here’s how it works:
Copy text from a photo
- Take a photo with text you want to copy into a document. For this example, we’re using a recipe from a cookbook.
- Go into the Photos app and select the photo with text you want to copy. This feature also works from the camera roll, inside the camera app.
- Press down on the text you want to copy with your finger, the same way you would in an Apple Note or text message. Two text selector dots should appear on the photo.
- Stretch the text selection cursors to cover all the text you want to copy. Press “copy.”
- Go into the word processor of your choice, tap on the text field, and paste.
Take a photo of text and put it directly into the Notes app
- The new feature also works directly in a word processor like the Notes app.
- Tap on the screen and pick the icon in the pop-up that looks like three lines of text in a square box. That’s the Live Text icon.
- It will bring a camera window up on the bottom half of the screen. Point it at text and the software will automatically recognize it.
- Apple’s software will highlight the text it wants to insert into the app inside the photo window.
- Press the blue “Insert” button when the software has selected the right chunk of text.
Scan text directly from the iPhone camera
- You can also scan text directly from the iPhone’s Camera app.
- Point the camera at words and if the software recognizes text, it will display the three-lines-of-text icon in the bottom right of the viewfinder, or top-right in landscape.
- Press the button. It will turn yellow and bring up a window dedicated to the text in the photo.
- You can select this text and copy-paste it into any document.