Apple has won a Major Patent for a Projection System for Macs and other devices like a Lamp to Capture and Project 3D Images & more

Today, Apple was granted a patent for a possible major projector device that could be added to Macs, a lamp, TV and more to project and capture 3D images and allow for 3D hand and finger gestures to control applications. This could be great for artists and gamers for starters.

In Apple's patent background they note that computers and other electronic devices may sometimes include displays. A display may present images to a user. With touch-sensitive displays, a user may interact with the images by providing touch input on the display.

Limitations may arise with traditional displays. For example, the user may wish to interact with real-world objects in the user's environment in addition to or instead of interacting with images on a display. Interacting with displayed images that have no connection to real-world objects may leave the user feeling removed from the user's real-world environment.

Electronic Devices With Projectors

Apple's granted patent covers an electronic device that may include a projector for creating the appearance of animated shadows on a surface. The animated shadows may be created by projecting ambient-light-matching illumination onto the surface that blends in with the surrounding ambient light.

Select pixels in the projector may be turned off so that one or more unilluminated regions are created within the surrounding illuminated region. The unilluminated regions may appear darker than the surrounding illuminated region, giving the appearance of a shadow. Characteristics of the shadow such as shape, size, and location may be adjusted by dynamically adjusting which pixels are turned off and which pixels are turned on to provide ambient-light-matching illumination.

The projector may be mounted in a housing such as a lamp housing, a furniture housing, a standalone projector housing, and/or any other suitable housing. The projector may be co-located with a position sensor that monitors positions of the surface, objects on the surface, a user or user's hands near the surface, and other objects.

The position sensor may be an array of optical emitters and detectors, one or more cameras (e.g., visible light cameras, stereoscopic imaging systems, infrared cameras, depth sensing cameras, etc.), one or more ultrasonic sensors, and/or one or more radio-frequency sensors such as ultra-wideband radio-frequency sensors.

A position sensor may also be used to gather user input such as gesture input (e.g., hand movements made near the surface, hand movements made with objects on the surface, hand movements made near the shadows, etc.). Control circuitry in the electronic device may adjust characteristics of the unilluminated regions based on voice input detected with the microphone, gesture input detected with the position sensor, and/or other sensor data.

The electronic device may include an ambient light sensor for measuring ambient light color and a user input device such as a microphone configured to gather user input.

Apple's patent FIG. 1 below is a schematic diagram of an illustrative system; FIG. 2 is a perspective view of an illustrative electronic device with a projector; FIG. 9 is a flow chart of illustrative steps involved in operating an electronic device with a projector that projects ambient-light-matching illumination onto a surface.

Apple's patent FIG. 3 below is a side view of an illustrative electronic device with one or more optical position sensors; FIG. 4 is a side view of an illustrative electronic device with one or more position sensors based on image sensor components; FIG. 5 is a side view of an illustrative electronic device with one or more ultrasonic position sensors; FIG. 6 is a side view of an illustrative electronic device with one or more position sensors based on radio-frequency components in accordance with an embodiment.

Apple's patent FIG. 7 above is a top view of a surface on which ambient-light-matching illumination is projected to create unilluminated regions with text shapes; FIG. 8 is a top view of a surface on which ambient-light-matching illumination is projected to create unilluminated regions with moving object shapes.

Games

Apple's patent makes special references to game playing. They note that control circuitry may dynamically adjust which pixels of projector are turned off and which pixels of the projector are turned on to adjust the shape, size, and location of unilluminated regions and thereby give unilluminated regions (sometimes referred to as shadows or apparent shadows) an animation effect.

Shadows may be used to present any suitable image such as images of objects (e.g., a boat, a battleship, a human, a bear, a shark, and/or any other suitable object), text, symbols, alphanumeric characters, moving images, patterns, rain effects, snow effects, mountains, ocean, buildings, floor plans, video game characters, movie or television characters, game pieces, game boards, and/or any other suitable image.

The control circuitry may adjust illuminated regions and unilluminated regions based on user input such as voice input detected by the microphone. For example, if the user is playing a game on the surface of a table and is speaking during the game, the microphone may detect the user's voice and control circuitry may use speech recognition techniques to determine what the user is saying. If voice input indicates that the user is building a battleship, the control circuitry may use the projector to create a battleship shadow using unilluminated regions.

This is one of those patents in which Apple is providing the basics of the invention without spelling out exactly what their true initial target application is for. Should the projector be an added accessor for an iMac, for instance, then it would somewhat resemble what the HP Sprout computer (Blended Reality PC), that was too ahead of its time.

(Click on image to Enlarge)

To confuse matters, Apple blurs the line beyond Macs and lists other possible future applications which includes applying the projector to an HMD, iPhone, iPad, TV, the Apple TV box, a lamp, a desk, sofa and more. Perhaps over time Apple will file for associated patents that will provide more details to shed more light as to the inventions true target device – even though a Mac seems to be the best application at this point in time.

For more details, review Apple's granted patent 11740689.

Posted by Jack Purcher on August 29, 2023 at 07:36 AM in 2. Granted Patents, Apple Accessories, Apple Macs , Devices, Components, In-Air Gesturing, Gestures, Eye-Tracking, Projection Related | Permalink | Comments (0)