Hello there readers,

This week’s update from our AI Feature Drops program brings Image Understanding to Aria. This new feature allows you to upload images to Aria in the sidebar chat, and ask questions about them. Image Understanding allows you to get information about an image while also providing context about it. 

Images are a crucial part of today’s web, and more often than not you will be searching for something related to a picture that you took yourself or that you found elsewhere. However, translating what’s depicted in an image to words can be really challenging – after all, an image says more than a thousand words, right?

For this reason, we have introduced Image Understanding to Aria, so that you can expand the ways in which you interact with content that you come across on a daily basis. Today we’ll show you how to use this powerful feature through some relatable examples.

To use this new feature, you will need to download the Developer stream of the browser and open the sidebar chat with Aria. Click the ‘+’ button just right of the chat input box and then you will find the ‘upload image’ option. You can upload up to 3 images at the same time and ask questions about them to Aria via text prompts.

Image Understanding gives you context about photographs

Say you are traveling through northern Europe, and during your stay in Norway you’re wandering through Oslo’s city center. You stumble upon what looks like an iconic building and take a picture of it. But, what is this building? From now on, you can get answers via Aria’s Image Understanding, just upload the image and ask about it. 

After using Image Understanding with Aria, we now know that the building we’re looking at is no other than the second most iconic Opera from Norway. Aria also gave us some context about the building, like when was it built and opened, who built it, and what is it used for.

Now, as you travel through northern Germany you will frequently see a landscape filled with metallic structures that resemble a fan. Curiosity gets the best of you and you take one of the pictures that you took from this trip and ask Aria about them. What could they be? Why put humongous fans in the middle of the fields?

A quick check with Aria brings some clarity to this mystery. Turns out that these ‘fans’ are actually turbines that harness the power of the wind to generate electricity from kinetic energy. A very ingenious invention that helps produce cleaner energy, and now we know all of this thanks to Aria’s Image Understanding, it’s fan-tastic!

Image Understanding can help you solve problems

Aria’s Image Understanding can also be a very helpful tool to solve different problems. For example, if you’re like me, and have issues with quickly solving math problems, you can use Aria’s Image Understanding to help you out.

In this case we gave Aria a hand-written equation and, through Image Understanding, it identified the problem and helped us solve it step by step. This is a nice example that showcases how Aria can interpret problems and help you solve them. 

But let’s not stop there, we want to test Aria’s capabilities even further. Let’s try and make Aria interpret more handwriting, and ask the browser AI to make a little Python program. 

Aria’s Image Understanding will read your handwriting and use your prompt in the chat to generate a few lines of code that will help you with your programming. This becomes useful because sometimes it’s difficult to translate your notes into proper code, especially if – like me– you’re just starting to learn the basics of coding.

Image Understanding + Image Generation, a boost to your creativity

Recently, we released an AI Feature Drop which gave Aria Image Generation capabilities. Today we’re releasing Image Understanding and we thought you’d like to know that the two features complement each other. 

You can provide an image to Aria, and request it to generate another one. We can give Aria one of my best drawings –don’t judge my artistic skills, please– as an example. When we upload it, we can request Aria in the prompt to create an image based on my drawing. Just test it to experience some magic happening IRL!

Here’s how this works: Aria uses Image Understanding to interpret what you want it to create and then uses Image Generation to bring it to life. The nice part about this feature is that it allows you to create images without necessarily having a long text prompt describing what you want. Instead, you can use a rough sketch and a short text prompt to get the image you want. 

Try Image Understanding today, download Opera

If you want to try this feature yourself, download Opera Developer today! Get access to our AI Feature Drop program and try out the latest experimental features that are being added on a biweekly basis. 

Don’t miss out on the latest AI news from Opera, scroll down and subscribe to our Newsletter down below.
________________________________________________________________________________________________________________
Images taken by the author of this post

AI Feature DropsAria AIOpera Developerupdate

---