Apple’s ‘Proactive’ to take on Google Now with deep iOS 9 search, Augmented Reality Maps, Siri API ~ Apple Juice

After several years of quiet development, Apple is readying a major new iOS initiative codenamed “Proactive,” which will leverage Siri, Contacts, Calendar, Passbook, and third-party apps to create a viable competitor to Google Now for Android devices. Like Google Now, Proactive will automatically provide timely information based on the user’s data and device usage patterns, but will respect the user’s privacy preferences, according to sources familiar with Apple’s plans.

As an evolution of iOS’s Spotlight search feature, Proactive is the fruit of a long-term initiative that involved the acquisition of small app developers, and integration of core iOS apps. It will also work with Apple’s Maps application to display personally relevant points of interest using an augmented reality interface, and integrate with a third-party Siri API codenamed “Breadcrumbs”…

How Spotlight Will Become A Google Now Rival

Apple began to lay the groundwork for Proactive with its acquisition of a personal assistant app called Cue in 2013, seeking to relevantly broaden iOS’s Spotlight and Safari search results. iOS 8’s ability to display Wikipedia Search results within Spotlight was the first taste of the Proactive initiative, and was partially designed to reduce iOS’s search reliance on Google. Sources say that Apple’s internal iOS usage metrics indicate that Google clicks have indeed fallen since iOS 8’s release last fall.

Now Apple wants to take Proactive to the next level, and it may do so with iOS 9’s introduction at the annual Worldwide Developers Conference on June 8. While Apple has positioned Siri as an “intelligent personal assistant” since the fall 2011 launch of the iPhone 4S, Proactive will go much further to integrate with your data. To begin with, Proactive will become a new layer within the iOS operating system, replacing the pulldown Spotlight menu currently found on the iOS Home screen.

Interestingly, Proactive will be found to the left of the first Home screen, just as Spotlight was prior to iOS 7’s major redesign. According to sources, Apple’s usage metrics for iOS suggest that Spotlight is not used as frequently as it could be because it is hidden above the Home screen without any indication of its location. Adding the feature back to the left of the Home screen will presumably boost usage, and consequently, take searches away from Google’s window within Safari. Sources note that some builds of iOS 9 allow Proactive to be accessed both with pulldown and left-of-Home screen gestures, so it’s possible that pulldown Spotlight-style access will be retained.

Cue app bought by Apple

Within the new Proactive screen, users will have a dedicated Search Bar at the top to access the established features of Spotlight: users will still be able to search for names, launch apps, and find audio tracks. Sources indicate that Proactive will include a greater emphasis on displaying news stories as search results, and will more reliably display news results when you search for information, including news on current topics and famous people.

Below the search bar will sit a new user interface that automatically populates with content based around three key parts of iOS: Apps, Contacts, and Maps, effectively a considerably upgraded version of Siri’s existing “digital assistant” functionality. For example, if a user has a flight listed in her Calendar application and a boarding pass stored in Passbook, a bubble within the new Proactive screen will appear around flight time to provide quick access to the boarding pass. If a user has a calendar appointment coming up, a map view could appear with an estimated arrival time, directions, and a time to leave indicator based on traffic. Proactive will also be able to trigger push notifications to help the user avoid missing calendar events. Even with these new notifications, however, the existing Notification Center apparently isn’t likely to see major changes.

Beyond Calendar integration, the feature will be able to integrate with commonly used apps. For example, if an iPhone user typically opens the Facebook app when he wakes up around 9AM, a button to access Facebook will start to appear for the user around 9AM. If the user calls his mother every Tuesday at 5PM, a bubble to “Call Mom” could appear around that time every Tuesday. As this feature integrates deeply with a user’s contact list, it is likely that the Recent Contacts menu introduced to the top of the Multitasking pane in iOS 8 will be relocated to Proactive’s interface. Lastly, Proactive will be able to display restaurant suggestions and ratings around breakfast, lunch, and dinner times in Proactive, changing based on the user’s location. Some of these features come from Apple’s acquisition of Spotsetter, a social-based search engine company founded by ex-Google Maps engineer Stephen Tse and Jonny Lee.

Yelp’s Monocle feature

Augmented Reality Maps, Browse Around Me

Beyond the new Spotlight interface, improvements to Maps will be a focal point for the Proactive initiative. In addition to a new public transportation service for bus, train, and subway routes, Apple is overhauling its Points of Interest (POI) system for the iOS Apple Maps application. While sources say Apple has explored new or expanded deals with Foursquare, Yelp, and Nokia’s HERE, the company ultimately decided to build its own new POI system for Proactive. Apple has developed a pair of new features around the POI system: an augmented reality view for local listings, as well as a feature dubbed “Browse Around Me.”

The augmented reality feature allows a user to hold up her iPhone in the Maps application, and point her camera toward a particular business or an area. Pointed towards a cafe, for example, the screen could show a virtual view of menu items or daily specials. If the user points her phone toward a street, a virtual outline of local businesses, restaurants, shopping stores, or coffee shops could appear. As a separate feature, the Browse Around Me button could simply show points of interest on an overhead map that are more tailored to previous searches and user preferences, a less flashy version of the augmented reality feature that’s similar to what Spotsetter was offering prior to its acquisition. Apple may choose to solely ship the Browse Around Me feature in iOS 9 and hold back the more extravagant augmented reality feature until later, the sources warned.

In order to develop this enhanced version of Maps into a small revenue source, Apple considered specifically highlighting points-of-interest that support Apple Pay transactions. However, this feature was cancelled amid development, according to a source with knowledge of the decision. Reports published during the initial launch of Apple Pay indicated that Apple takes 0.15% of every Apple Pay transaction conducted via an iPhone or Apple Watch.

“Breadcrumbs:” A Limited Siri API To Address User Privacy Concerns

Apple does not want to launch Proactive solely with support for its own apps and data. According to sources, Apple is preparing to allow developers to integrate their applications into both iOS’s search results and Siri, which has been redesigned to be more colorful in iOS 9, akin to its Apple Watch interface. While Apple has worked on a full Siri API that lets Siri access third-party app content, sources say the company is more likely to introduce a scaled-down service for developers codenamed “Breadcrumbs.” The decision is apparently based on a concern that a deeply integrated Siri will misinterpret words or contexts, sending personal data to the wrong third-party apps, and creating privacy issues, according to the sources.

Breadcrumbs allows Proactive and Siri to index parts of apps that have recently been accessed, and is apparently similar to the App Indexing feature Google announced previously for Android, and today for iOS. For instance, if a user typically watches certain genres of movies in the Netflix app, the ability to launch that particular section in the Netflix app could be opened by Siri. Eventually, Apple hopes to release a full Siri API for developers, but the company will not do so until all potential privacy concerns are sorted out. As one source said, “Apple’s talk about user-privacy is not marketing, it is core to [Apple’s] product development process.”

Executives Mixed on Rollout

While all of the aforementioned features are far into development, none of them as described are a lock for iOS 9.0 due to Apple’s new focus on maintaining (and improving) quality in iOS and OS X. According to sources, Vice President of Product Management for iOS and OS X Kim Vorrath has been reluctant to launch the new Proactive initiative with such an ambitious set of features.

Apple would rather launch scaled down incarnations of the aforementioned enhancements and then improve the feature over time, according to the sources. Nonetheless, these features are currently in development within Apple’s Software Engineering department, and the scale at which they launch will become clear at Apple’s June 8th Worldwide Developers Conference kickoff keynote. Sources indicate that Apple’s marketing department will determine whether the features are launched under the Proactive umbrella name, or described as upgrades to existing features such as Spotlight, Maps, Contacts, Calendars, and Siri.

In addition to developing its new Proactive features, Apple is working on a series of other updates for iOS 9. As we have reported, Apple is developing a new streaming music service based on its acquisition of Beats Electronics, a new Home app for managing home automation accessories, a refreshed look for iPhones and iPads with the new San Francisco typeface, new security and optimization features, iPad split-screen apps, a new keyboard, and iMessage improvements.