Tag Archives: LIDAR

Learn More About Triggering Augmented Reality Experiences with AR Markers

learns an markers-area and play it-chrome

We expect a continued increase in the utilization of AR in 2021. The iPhone 12 contains LiDAR technology, which enables the use of ARKit 4, greatly enhancing the possibilities for developers. When creating an AR application, developers must consider a variety of methods for triggering the experience and answer several questions before determining what approach will best facilitate the creation of a digital world for their users. For example, what content will be displayed? Where will this content be placed, and in what context will the user see it?

Markerless AR can best be used when the user needs to control the placement of the AR object. For example, the IKEA Place app allows the user to place furniture in their home to see how it fits.

1_0RtFp6lxeJWxcg5EE_wYCg

Location-based AR roots an AR experience to a physical space in the world, as we explored previously in our blog Learn How Apple Tightened Their Hold on the AR Market with the Release of ARKit 4. ARKit 4 introduces Location Anchors, which enable developers to set virtual content in specific geographic coordinates (latitude, longitude, and altitude). To provide more accuracy than location alone, location anchors also use the device’s camera to capture landmarks and match them with a localization map downloaded from Apple Maps. Location anchors greatly enhance the potential for location-based AR; however, the possibilities are limited within the 50 cities which Apple has enabled them.

Marker-based AR remains the most popular method among app developers. When an application needs to know precisely what the user is looking at, accept no substitute. In marker-based AR, 3D AR models are generated using a specific marker, which triggers the display of virtual information. There are a variety of AR markers that can trigger this information, each with its own pros and cons. Below, please find our rundown of the most popular types of AR markers.

FRAMEMARKERS

5fc9da7d2761437fecd89875_1_gXPr_vwBWmgTN5Ial7Uwhg

The most popular AR marker is a framemarker, or border marker. It’s usually a 2D image printed on a piece of paper with a prominent border. During the tracking phase, the device will search for the exterior border in order to determine the real marker within.

Framemarkers are similar to QR Codes in that they are codes printed on images that require handheld devices to scan, however, they trigger AR experiences, whereas QR codes redirect the user to a web page. Framemarkers are a straightforward and effective solution.

absolut-truths

Framemarkers are particularly popular in advertising applications. Absolut Vodka’s Absolute Truth application enabled users to scan a framemarker on a label of their bottle to generate a slew of more information, including recipes and ads.

GameDevDad on Youtube offers a full tutorial of how to create framemarkers from scratch using Vuforia Augmented Reality SDK below.

 

NFT MARKERS

?????????

NFT, or Natural Feature Tracking, enable camera’s to trigger an AR experience without borders. The camera will take an image, such as the one above, and distill down it’s visual properties as below.

AugementedRealityMarkerAnymotionFeatures

The result of processing the features can generate AR, as below.

ImEinsatz

The quality and stability of these can oscillate based on the framework employed. For this reason, they are less frequently used than border markers, but function as a more visually subtle alternative. A scavenger hunt or a game employing AR might hide key information in NFT markers.

Treasury Wine Estates Living Wine Labels app, displayed above, tracks the natural features of the labels of wine bottles to create an AR experience which tells the story of their products.

OBJECT MARKERS

image1-7

The  toy car above has been converted into an object data field using Vuforia Object Scanner.

image4-1

Advancements in technology have enabled mobile devices to solve the issue of SLAM (simultaneous localization and mapping). The device camera can extract information in-real time, and use it to place a virtual object in it. In some frameworks, objects can become 3D-markers. Vuforia Object Scanner is one such framework, creating object data files that can be used in applications for targets. Virtual Reality Pop offers a great rundown on the best object recognition frameworks for AR.

RFID TAGS

Although RFID Tags are primarily used for short distance wireless communication and contact free payment, they can be used to trigger local-based virtual information.

While RFID Tags are not  widely employed, several researchers have written articles about the potential usages for RFID and AR. Researchers at the ARATLab at the National University of Singapore have combined augmented reality and RFID for the assembly of objects with embedded RFID tags, showing people how to properly assemble the parts, as demonstrated in the video below.

SPEECH MARKERS

Speech can also be used as a non-visual AR marker. The most common application for this would be for AR glasses or a smart windshield that displays information through the screen requested by the user via vocal commands.

CONCLUSION

Think like a user—it’s a staple coda for app developers and no less relevant in crafting AR experiences. Each AR trigger offers unique pros and cons. We hope this has helped you decide what is best equipped for your application.

In our next article, we will explore the innovation at the heart of AIoT, the intersection of AI and the Internet of Things.

Learn How Apple Tightened Their Hold on the AR Market with the Release of ARKit 4

arkit-og

Since the explosive launch of Pokemon Go, AR technologies have vastly improved. Our review of the iPhone 12 concluded that as Apple continues to optimize its hardware, AR will become more prominent in both applications and marketing.

At the 2020 WWDC in June, Apple announced ARKit 4, their latest iteration of the famed augmented reality platform. ARKit 4 features some vast improvements that help Apple tighten their hold on the AR market.

LOCATION ANCHORS

ARKit 4 introduces location anchors, which allow developers to set virtual content in specific geographic coordinates (latitude, longitude, and altitude). When rebuilding the data backend for Apple Maps, Apple collected camera and 3D LiDAR data from city streets across the globe. ARKit downloads the virtual map surrounding your device from the cloud and matches it with the device’s feed to determine your location. The kicker is: all processing happens using machine learning within the device, so your camera feed stays put.

36431-67814-ARKit-xl

Devices with an A12 chip or later, can run Geo-tracking; however, location anchors require Apple to have mapped the area previously. As of now, they are supported in over 50 cities in the U.S. As the availability of compatible devices increases and Apple continues to expand its mapping project, location anchors will find increased usage.

DEPTH API

ARKit’s new Depth API harnesses the LiDAR scanner available on iPad Pro and iPhone 12 devices to introduce advanced scene understanding and enhanced pixel depth information in AR applications. When combined with 3D mesh data derived from Scene Geometry, which creates a 3D matrix of readings of the environment, the Depth API vastly improves virtual object occlusion features. The result is the instant placement of digital objects and seamless blending with their physical surroundings.

FACE TRACKING

1_tm5vrdVDr2DAulgPvDMRow

Face tracking has found an exceptional application in Memojis, which enables fun AR experiences for devices with a TrueDepth camera. ARKit 4 expands support to devices without a camera that has at least an A12. TrueDepth cameras can now leverage ARKit 4 to track up to three faces at once, providing many fun potential applications for Memojis.

VIDEO MATERIALS WITH REALITYKIT

b3b1c224-5db5-4e38-97de-76f90c32b53a

ARKit 4 also brings with it RealityKit, which adds support for applying video textures and materials to AR experiences. For example, developers will be able to place a virtual television on a wall, complete with realistic attributes, including light emission, texture roughness, and even audio. Consequentially, AR developers can develop even more immersive and realistic experiences for their users.

CONCLUSION

iOS and Android are competing for supremacy when it comes to AR development. While the two companies’ goals and research overlap, Apple has a major leg up on Google in its massive base of high-end devices and its ability to imbue them with the necessary structure sensors like TrueDepth and LiDAR.

ARKit has been the biggest AR development platform since it hit the market in 2017. ARKit 4 provides the technical capabilities tools for innovators and creative thinkers to build a new world of virtual integration.

How App Developers Can Leverage the iPhone 12 to Maximize Their Apps

iPhone 12

On October 23rd, four brand new iPhone 12 models were released to retailers. As the manufacturer of the most popular smartphone model in the world, whenever Apple delivers a new device its front-page news. Mobile app developers looking to capitalize on new devices must stay abreast of the latest technologies, how they empower applications, and what they signal about where the future of app development is headed.

With that in mind, here is everything app developers need to know about the latest iPhone models.

BIG DEVELOPMENTS FOR AUGMENTED REALITY

LiDAR is a method for measuring distances (ranging) by illuminating the target with laser light and measuring the reflection with a sensor

LiDAR is a method for measuring distances (ranging) by illuminating the target with laser light and measuring the reflection with a sensor

On a camera level, the iPhone 12 includes significant advancements. It is the first phone to record and edit Dolby Vision with HDR. What’s more, Apple has enhanced the iPhone’s LiDAR sensor capabilities with a third telephoto lens.

The opportunities for app developers are significant. For AR developers, this is a breakthrough—enhanced LiDAR on the iPhone 12 means a broad market will have access to enhanced depth perception, enabling smoother AR object placement. The LIDAR sensor produces a 6x increase in autofocus speed in low light settings.

The potential use cases are vast. An enterprise-level application could leverage the enhanced camera to show the inner workings of a complex machine and provide solutions. Dimly lit rooms can now house AR objects, such as Christmas decorations. The iPhone 12 provides a platform for AR developers to count on a growing market of app users to do much more with less light, and scan rooms with more detail.

The iPhone 12’s enhanced LiDAR Scanner will enable iOS app developers to employ Apple’s ARKit 4 to attain enhanced depth information through a brand-new Depth API. ARKit 4 also introduces location anchors, which enable developers to place AR experiences at a specific point in the world in their iPhone and iPad apps.

With iPhone 12, Apple sends a clear message to app developers: AR is on the rise.

ALL IPHONE 12 MODELS SUPPORT 5G

5G 2

The entire iPhone 12 family of devices supports 5G with both sub-6GHz and mmWave networks. When iPhone 12 devices leverage 5G with the Apple A14 bionic chip, it enables them to integrate with IoT devices, and perform on ML algorithms at a much higher level.

5G poses an endless array of possibilities for app developers—from enhanced UX, more accurate GPS, improved video apps, and more. 5G will reduce dependency on hardware as app data is stored in the cloud with faster transfer speeds. In addition, it will enable even more potential innovation for AR applications.

5G represents a new frontier for app developers, IoT, and much more. Major carriers have been rolling out 5G networks over the past few years, but access points remain primarily in major cities. Regardless, 5G will gradually become the norm over the course of the next few years and this will expand the playing field for app developers.

WHAT DOES IT MEAN?

Beyond the bells and whistles, the iPhone 12 sends a very clear message about what app developers can anticipate will have the biggest impact on the future of app development: AR and 5G. Applications employing these technologies will have massive potential to evolve as the iPhone 12 and its successors become the norm and older devices are phased out.