With iOS 18, the latest update to the iPhone's operating system, Apple will introduce the biggest and most radical redesign of the Photos app ever shipped since the iPhone's inception. The new version of the software is built around a fully reorganized UI, with more space for collections and memories created both by the iPhone—thanks to some clever use of machine learning—and by the users, manually or by employing natural language. 

To better understand the reasons and philosophy behind the redesign, we spoke with Apple's Della Huff, from the Camera and Photos Product Marketing team; Billy Sorrentino, Senior Director for Human Interface Design in Apple's design team; and Jon McCormack, Vice President with the Camera and Photos Software Engineering team and nature photographer.

We asked them what motivated such a radical rethinking of the Photos app experience, how they approached the design and engineering effort, and how this new app fits with iOS 18's newfound "customization-friendly" approach.

The revamped Photos app introduces a suite of features, including a more streamlined navigation, enhanced organizational tools, and new ways to rediscover cherished memories of friends, family, pets, and travels. In the new default app view, the top section shows the familiar photo grid for chronological browsing, with an added filter function to refine content types and exclude screenshots. Thanks to iOS 18's AI capabilities, the app now also curates collections based on topics such as time, people, and trips, with a "Recent Days" feature that intelligently filters out clutter and presents an auto-playing view of daily highlights. Users can also easily swipe between recent collections and share photo compilations with a tap. Moreover, a new grouping feature assembles photos group photos of the user with friends, or family members, making it easier to locate and enjoy memories that include our loved ones. 

Jon McCormack

"With the new Photos in iOS 18, we've created a simple, streamlined, single view photos experience based all around deep intelligence", said Billy Sorrentino. "Ultimately, we wanted to remove friction and help users find the photos they want and show them in a delightful and cinematic way. Collections were imagined to be a lean back, media rich experience, with live photos autoplaying the best part of each clip and seamlessly integrating with all the other pictures".

With the redesign, the Photos app is now aspiring to be more than just a virtual drawer where all our pictures end up in a disorganized heap. 
"The idea is that the app can highlight your best content, your favorites, your featured photos, your memories," says Sorrentino. "The goal is for Photos to become more serendipitous while surprising and delighting the user with its ability to understand what content is most important to the user."

Yet, the Photos app is still also the virtual destination of any media we generate with our iPhone's camera or screenshot feature. While photos can be memories, they're also very often random bits and pieces of our visual lives. That's why the new Photos app introduces new and more effective filters that can exclude different kinds of pictures from the main view based on their type or content.

"Today, over 3 trillion photos and videos a year shot with iPhone, which is the world's most popular camera", says Della Huff. "Our camera does many jobs. It doesn't just capture precious memories but also receipts and documents, where we parked the car, what we had for lunch, all of those kinds of more utilitarian things that we also need to take photos of". By creating filters that more effectively discern between the "memory and photos" content and the "utilitarian content," says Huff, Apple wanted to "make it easier for people to find what they're looking for, so they can spend less time searching for things and more time actually enjoying the content that matters to them." 

To achieve these goals, the design and engineering team leveraged on-device intelligence (i.e., machine learning and deep learning) like never before. It's fair to say that this is the most "intelligent" version of Photos Apple has ever shipped on an iPhone. Most of the features previewed so far, revolve one way or another around the ability of the UI to integrate with the ability of the system to "understand" the pictures, their content, and their quality. 

"The Photos app has been a leader in on-device intelligence for about 15 years.. We used it and we use it to recognize people and pets in your photos and create those beautiful memory videos of significant people and events in your life," says Jon McCarmack from the Camera Software Engineering team. "The goal of this intelligence, of course, is to make your life easier. And that is what the intelligent features in the new Photos are all about".

In the Recent Days collections, for example, the iPhone automagically selects the best pictures from a specific day to create a deeply personal and emotional memory of an event. For the first time, the app can also organize photos of groups of people and people with pets. Some groups will be created automatically, but users can also add groups manually based on the data collected by the ML algorithms within the app. 

"Another great feature is the trips collection, which helps you relive those memorable moments," adds McCormack. "The iPhone automatically creates these slideshows for you so that you can sit back and enjoy or share them with the people who lived those moments with you.

Of course, everything happens and stays only on each user's phone, respecting their privacy.

"I've used the words on device intelligence a couple of times, and that's really important," says McCormack. "We know your photos are extremely personal, so they're only ever processed on your device. Apple doesn't access your photos and videos and doesn't ever use them for any kind of research and development. They're yours and yours alone."

Many of the automatic phone selections powered by AI revolve around the device's ability to discern and select the best pictures for each given collection. But what do we exactly mean by "best," and who teaches the machine what to select and how?

"There is no definitive answer on this, as it's still an ongoing journey," says McCormack. "Nonetheless, there are tons of signals we understand about the content inside your library, and we use those to craft the entire experience.

We start with the obvious technical aspects:' Is it a decent picture? Are the eyes open? Is the person smiling? Is it a sharp photograph', etcetera, etcetera. 
On-device intelligence can also help you figure out who your important people are, and that's usually a very strong indicator that those are the pictures that are most important to you."

By putting together all these indicators and understanding the relationship between the photos and the subject, using external on-device metadata such as time, location, contacts, and more, the app can understand what the "most significant" pictures are for each user.

"It's not necessarily a value judgment on the best photograph per se. It's a value judgment on for what we want this photograph to do right now", explains McCormack, "which is to delight you on the photos widget as you're on your way to some other app or presenting you a surprising collection that helps you relive a beautiful memory, and so on." 

One point that wasn't clear to us is why these tools should not be used to help users trim down their excessively sized libraries. If the iPhone can understand what pictures matter most to us, why can't we get rid of the others?

"Because you'll never know when a picture might become important to you. The whole point of this redesign is exactly that: stop the doom scrolling so that you can keep all the photos you ever took and never have to throw anything away. But we'll get you to where you want to get to as quickly as possible, filtering and selecting for the content that really matters".

As in previous radical redesigns, the new Photos has received somewhat divisive feedback from its user base. The most vocal critics decry a departure from the simplicity of the previous Photos app, while the enthusiasts cheer for the complete overhaul of the former UI.

What's interesting with the new Photos is that the ability to customize the experience to an almost non-Apple degree is also baked in. The entire view with the grids and collections can be reorganized and personalized completely. The goal was to create an app that, despite having to deal with billions of different photo libraries, can dynamically adapt to every user's needs. 

"No matter how big your photo library is, you might be somebody who really cares about the photos of your dog or your children and the people in your life," says Huff. "With this new design, you can elevate those to the carousel or make those pinned collections right at your fingertips."

The goal aligns with the mantra of full-on customization that underpins the entire iOS 18 launch, perfectly summed up by the snazzy slogan Apple picked for its marketing preview — "Yours. Truly". 

"Because we know we've got so many different types of Photos users, especially with the heritage of this app and our global scale," says Sorrentino," we wanted to make sure that photos are your most personal content, becomes your most personal app. Once the new app will be out, it'd be really delightful to see how different it will adapt to become on each and every users' iPhone, feeling customized and personal to each one of them".