(Nanowerk News) A challenge to adopting augmented reality (AR) in wider applications is working with dynamic objects, owing to a delay between their movement and the projection of light onto their new position. But, Tokyo Tech scientists may have a workaround. They have developed a method that uses multiple projectors while reducing delay time. Their method could open the door to a future driven by AR, helping us live increasingly technology-centered lives.
Technological advancements continue to redesign the way we interact with digital media, the world around us, and each other. Augmented reality (AR), which uses technology to alter the perception of objects in the real world, is unlocking unprecedented landscapes in entertainment, advertising, education, and across many other industries.
The complex system integrates high-speed cameras and a sensing computer which is connected to a network hub. The network is then sent to several computers, all connected to their respective projector which illuminates the target. This way, the addition of more projectors does not have a large negative effect of the latency between camera sensing and projection. (Image: Yoshihiro Watanabe)
The use of multiple projectors plays an important role in expanding the usage of AR, alongside a technique called projection mapping. However, an obstacle to the widespread adoption of AR is the application of this method to moving, or “dynamic,” targets without the loss of immersion in the AR space.
This technique, known as dynamic projection mapping, relies on a blend of cameras and projectors that visually detect target surfaces and project onto them, respectively. A critical aspect is the need for high speed in information transfer and low “latency,” or delay between detection and projection. Any latency leads to a misalignment of the projector, which effects our perception, and reduces the effectiveness of the AR space.
The weight, or importance, of a given pixel is used to control the projector intensity on each pixel. The weights are assigned depending on the area of the surface that is illuminated. Pixels with a smaller projection area on the surface are controlled to have larger weight, whereas pixels with a larger area are controlled to have smaller weight. (Image: Yoshihiro Watanabe)
Other issues like changes in shadowing and target overlap are solved easily by using multiple projectors. However, the addition of new projectors correspondingly drives up the latency. This is a result of the need to calculate the intensity at every pixel simultaneously for every frame of a moving scene. Simply put, more projectors lead to longer and more complex calculations. The latency is a massive hurdle towards AR taking a true foothold in broader applications across society.
Thankfully, a team of scientists at Tokyo Institute of Technology (Tokyo Tech), led by Associate Professor Yoshihiro Watanabe, might just have the necessary solution. They have developed a novel method to calculate the intensity of each pixel on a target in parallel, reducing the need for a single large optimization calculation. Their method relies on the principle that if pixels are small enough, they can be evaluated independently.
While based on an approximation, their results, published in IEEE Transactions on Visualization and Computer Graphics (“Dynamic Multi-projection Mapping Based on Parallel Intensity Control”), suggest that they could achieve the same quality of images as conventional, more computationally expensive methods, while drastically increasing the mapping speed and thereby reducing the latency.
(Left) A single projector cannot eliminate the shadow caused by the moving object at the left on the projection target placed in the center. (Right) A newly developed multi-projector system consisting of four units can eliminate shadows even in dynamic scenes. (Image: Yoshihiro Watanabe)
“Another advantage of our proposed method is, as there is no longer need for a single global calculation, it allows the use of multiple rendering computers connected through a network, each only controlling a single projector,” explains Dr. Watanabe. “Such a network system is easily customizable to incorporate more projectors, without major sacrifices to the latency.”
This new method can allow large spaces with many projectors for efficient dynamic projection mapping, taking us a step closer to broader AR applications, as Dr. Watanabe describes: “The presented high-speed multi-projection is expected to be a major part of important base technologies that will advance spatial AR to derive more practical uses in our daily life.”
