skill: add Android skills

This commit is contained in:
2026-07-14 10:34:54 +08:00
parent e260c02629
commit ab82fb7cf0
182 changed files with 62992 additions and 0 deletions

View File

@@ -0,0 +1,54 @@
Developing camera features for XR devices, headsets, and AR glasses requires a
shift from 2D pixel-pushing to 3D spatial awareness.
## Understand the XR development mindset
| Concept | Mobile focus | **XR focus** |
|---|---|---|
| **Input** | Raw camera stream | **Spatial tracking (visual-inertial odometry (VIO) or simultaneous localization and mapping (SLAM))** |
| **Output** | Screen viewfinder | **Stereo passthrough and occlusion** |
| **Constraint** | Battery life | **Motion-to-photon latency (less than 20 ms)** |
*** ** * ** ***
## Follow the implementation guide
### API selection
On XR devices, standard `CameraX` implementations are often restricted or
insufficient. Always use spatial software development kits (SDKs):
- **ARCore**: Use ARCore for plane detection, depth sensing, and motion tracking.
- **OpenXR**: Use OpenXR as the cross-platform standard for VR and AR rendering and input.
- **OEM SDKs**: Use manufacturer-specific libraries for hardware-accelerated passthrough.
### Handle spatial passthrough
Unlike a 2D viewport, XR passthrough is often system-managed.
**\[Key requirement\] Frame synchronization**: Synchronize your application's
frame clock with the headset's head-mounted display (HMD) pose.
```kotlin
// Example: Querying the spatial pose for the current camera frame
val headPose = xrSession.getHeadPose(frameTime)
val projectionMatrix = headPose.getProjectionMatrix(eyeIndex)
```
<br />
### Manage depth and occlusion
Digital content must respect real-world depth to ensure accurate occlusion.
- **Depth map** : Access raw depth data using `ARCore` or `SurfaceProcessor` to create an occlusion mask.
- **Hardware buffers** : Use `HardwareBuffer` to share camera frames directly with the GPU without CPU-side copies to minimize latency.
*** ** * ** ***
## XR pitfalls
- **The nausea limit**: Any processing that delays the viewfinder by more than 20 ms causes user sickness. Don't perform image processing on the main thread.
- **Privacy restrictions** : Some XR devices return a black frame if you attempt to record the "Passthrough" layer. Check `Session.isRecordingSupported`.
- **Field of view (FOV)**: The camera FOV possibly doesn't match the display FOV. Use the SDK's projection matrixes instead of calculating aspect ratios manually.
- **Front buffer rendering** : If the device supports it, use `FrontBufferRenderer` for real-time overlays to bypass standard double-buffering latency.