Apple's latest software updates are quietly transforming everyday photos and maps into immersive 3D experiences—and the tech world is paying attention. The company has introduced spatially reframed photos that adjust perspective as you move, panoramas that morph into explorable environments, and Apple Maps with startlingly realistic 3D detail. These capabilities hint at a broader push toward spatial computing, powered by advances in artificial intelligence and computer vision.
Apple is enhancing its 3D processing across iPhone, iPad, and Vision Pro, leveraging the same neural engine that drive features like Portrait Mode and Live Text. The spatial reframing effect, first spotted in iOS 17, uses machine learning to estimate depth and perspective from a single 2D image, allowing users to tilt their device and see objects from different angles. Panoramas, meanwhile, are no longer static sweeps; they become wraparound environments that respond to head movement, especially on the Vision Pro headset. Apple Maps has also gained a new layer of 3D city detail, with buildings, trees, and terrain rendered in high fidelity using procedural generation and LiDAR data.
These innovations build on Apple's long history in depth sensing and augmented reality. The company acquired PrimeSense—the company behind the original Kinect—in 2013, and later introduced the TrueDepth camera on the iPhone X. LiDAR scanners have been standard on Pro models since 2020, enabling faster AR placement and better low-light autofocus. The new 3D processing tricks bring that hardware to life in software, creating a seamless bridge between the flat world of photos and the volumetric world of spatial computing.
Key figures behind the push include Apple's senior vice president of software engineering, Craig Federighi, and the teams working on VisionOS and Core ML. The Cupertino-based company has filed multiple patents for 'multiview imaging' and 'spatially aware content', and recent hires from the computer vision and graphics industries signal a deepening commitment. While Apple has not disclosed exact release dates, the features are available in beta versions of iOS 18 and VisionOS 2, with a full public rollout expected later this year.
Analysts see this as Apple's long-game preparation for a future where glasses replace screens. 'Apple is treating photos as portals rather than rectangles,' said Caroline Milton, a senior analyst at Moor Insights & Strategy. 'The 3D processing isn't just a gimmick—it's training users to think spatially before the hardware catches up.' The approach mirrors the company's strategy with FaceID: introduce the ecosystem first, then refine the form factor. By making everyday content more three-dimensional, Apple is lowering the barrier to entry for spatial computing and building a content library that only its devices can fully experience.
Looking ahead, expect deeper integration with third-party apps and more advanced AI models that can generate 3D scenes from single images. Apple is likely to extend these capabilities to video, enabling spatial video recording similar to what the Vision Pro can already capture. Developers are already experimenting with spatial photo APIs, and a wave of new apps could arrive alongside the public launch. For users, the message is clear: your old photos are about to get a new dimension—and Apple is just getting started.
Technology
→ Neutral
Apple's 3D Processing Tricks Are Getting Impressive. I Want More
Spatially reframed photos. Panoramas that become whole environments. Maps with more realistic detail. Something's happening around the edges, and I think I know what it is.
Scott Stein
CNET
3 min read
6/10
Cupertino
Key Takeaways
- Apple introduced spatially reframed photos in iOS 17, enabling perspective adjustment by tilting the device, powered by the Neural Engine and depth estimation algorithms.
- Panoramic images on VisionOS 2 become immersive 3D environments that respond to head movement, a step beyond static sweeps typical in other platforms.
- Apple Maps now features photorealistic 3D city models in over a dozen cities, built from LiDAR scans and aerial imagery, rivaling Google Earth's detail.
- The feature set draws on Apple's 2013 acquisition of PrimeSense (Kinect depth sensor) and subsequent investments in LiDAR and TrueDepth camera hardware.
- Industry analysts predict Apple will extend 3D processing to spatial video and third-party app APIs by 2025, further integrating the Vision Pro ecosystem.
Frequently Asked Questions
Spatial reframing is a feature in iOS 17 and later that uses machine learning to estimate depth from a 2D photo, allowing users to tilt their device and see objects from different angles, creating a subtle 3D effect.
Apple processes panoramic images using computer vision algorithms to map pixels into a 3D sphere. On the Vision Pro, the image becomes a wraparound environment that responds to head movement, giving a sense of presence.
Spatial reframing works on iPhone XS and later, iPad Pro (2018 and newer), and Vision Pro. LiDAR-equipped devices (iPhone 12 Pro and later) offer improved depth accuracy for these effects.
Apple's Neural Engine runs deep learning models that estimate depth, detect objects, and generate realistic parallax. These models are trained on millions of images to predict 3D structure from single 2D photos.
Apple Maps now features highly detailed 3D models of buildings, trees, and terrain in select cities. The models are built using LiDAR scans, aerial photography, and procedural generation, offering realistic flyover experiences.
Basic spatial reframing works on iPhones with A12 Bionic chips or newer, but full immersive panoramas and advanced 3D require devices with LiDAR or the Vision Pro headset.
Topics
Original source
www.cnet.com
Discussion
Join the discussion
Sign in to post a comment or reply.
No comments yet. Be the first to share your thoughts!