Building from Scratch: Fieldwire's New 360° Photo Viewer
Many users rely on Fieldwire’s support for 360° photos to easily capture and communicate the status of their job site when working in the field. Late last year, we shipped an all-new version of the 360° photo viewer in Fieldwire’s iOS app. However, from the user’s perspective, there weren’t any noticeable changes in functionality. Instead, the goal was to replace the old viewer with a new implementation tailored to our specific needs and which would be easier to maintain long-term.
The original version of Fieldwire’s viewer was powered by the Google VR SDK for iOS.
It worked well for a number of years, but was eventually deprecated and stopped receiving updates. As a result, it
contained the last remaining references to UIWebView in the app, which became a problem since UIWebView will soon be
disallowed in new app updates. A few months ago, we began looking for an
open source alternative. Unfortunately, many of the projects we evaluated had their own downsides. Some included features we
didn’t need, like stereoscopic rendering for VR. Others brought in lots of additional dependencies that impacted binary size,
or only worked on versions of iOS newer than our minimum deployment target of iOS 11. All of these problems could be worked around,
and some of them weren’t new either; the old viewer’s dependencies already impacted the app’s size. That being said, we wanted the
new viewer implementation to be an improvement over the old one, so we decided to investigate what it would take to build our own
from scratch, tailored to Fieldwire’s specific feature set.
As a result of this work, we recently shipped the new Fieldwire 360° photo viewer, which runs at 60fps on a wide range of devices and has zero 3rd party dependencies, all in less than 200 lines of code. We were able to accomplish this by leveraging Apple’s SceneKit and CoreMotion frameworks for rendering and orientation detection respectively.
Fieldwire stores 360° photos as ordinary JPEG images using equirectangular projection, which represents the spherical image as a distorted rectangle with a 2:1 aspect ratio. To render part of the image using SceneKit, we apply it as a material to the interior surface of a sphere, placing a camera at the center. Then, the camera can be rotated as needed to view any part of the image. This simple scene can be constructed in only a few lines of code using SceneKit, avoiding much of the cognitive overhead of alternative rendering libraries. In our tests, it also performed well across a wide range of devices, and integrated well with Xcode’s debugging tools. By leveraging SceneKit instead of following the approach of alternatives that relied on OpenGL or Metal to support more advanced use cases, we were able to ship our own viewer without significantly increasing the complexity of our codebase.
Intuitive controls were another important consideration when designing the new viewer. In addition to using horizontal swipes to rotate the view of the photo around the vertical axis, motion controls can also be used to change the pitch, roll, and yaw of the virtual camera. Here, we relied on CoreMotion’s sensor fusion to combine accelerometer, gyroscope, and magnetometer data into a smooth, precise measurement of device orientation. This measurement can be combined with the touch inputs to express the desired camera orientation as a quaternion, avoiding gimbal lock. As a result, it’s easy to use either input method to look around the scene without jittering. Again, here we were able to leverage system frameworks to avoid introducing excessive complexity during the rewrite.
Ultimately, by taking a step back and reviewing our options when we were forced to migrate away from our old viewer implementation, we were able to make a number of engineering improvements beyond just getting rid of the most pressing technical debt. By leveraging builtin frameworks instead of third party dependencies, overall app size was reduced by around 5mb. The new implementation is also much easier for engineers to understand and maintain because it only includes the specific features our app’s users need. That’s not to say that rewrites to remove 3rd party dependencies are always the right solution. In fact, if not done carefully they can just result in unnecessary work and a larger codebase to manage. In this case though, the significant benefits far outweigh the disadvantages for a relatively straightforward feature like this one. The end result was a new, modern implementation of the 360° photo viewer which we expect will serve us well for many years to come. We’re already thinking about great new features it could enable, like the ability to easily share 2D snapshots of a 360° scene or add information via markups!