Picking System

Live Demo: Picking

Picking is the problem of answering "what entity is under the cursor, and where on its surface is the cursor hitting." The CPU answer is to raycast through the scene. Build a ray from the camera through the cursor pixel, intersect it against every entity's bounding volume to get candidates, then against every triangle of every candidate for the closest hit. The math is well understood and the cost scales with scene size and triangle count.

Nightshade ships two picking paths. The fast one tests rays against bounding volumes. The precise one rasterizes triangle meshes through Rapier's SharedShape::trimesh and casts rays against them. Both produce a PickingResult with entity, distance, and world position. There is also a GPU-driven path that reads the entity-id and depth textures the renderer is already writing, covered in the picking article and in the renderer source.

Screen-to-ray conversion

PickingRay::from_screen_position converts a 2D screen coordinate into a 3D ray. It computes NDC coordinates from the screen position, builds the inverse view-projection of the active camera, and unprojects through it.

For a perspective camera, the ray origin is the camera position. A clip-space point at z = 1.0 (the reversed-Z near plane) unprojects to the world direction.

For an orthographic camera, both near (z = 1.0) and far (z = 0.0) clip-space points are unprojected. The ray origin is the near point and the direction is the vector from near to far.

Viewport rectangles are handled by converting screen coordinates to local viewport space and scaling by the viewport-to-window ratio.

#![allow(unused)]
fn main() {
pub struct PickingRay {
    pub origin: Vec3,
    pub direction: Vec3,
}

let screen_pos = world.resources.input.mouse.position;
if let Some(ray) = PickingRay::from_screen_position(world, screen_pos) {
    // ray.origin and ray.direction are in world space
}
}

Bounding volume picking

The fast path tests the ray against every entity's bounding volume. For each entity with a BoundingVolume component, the routine transforms the bounding volume by the entity's global transform, early-rejects with a bounding-sphere test (project center onto ray, check distance), then tests the OBB for a precise intersection distance. Invisible entities are skipped via the Visibility component by default.

Results are sorted by distance, closest first.

#![allow(unused)]
fn main() {
if let Some(hit) = pick_closest_entity(world, screen_pos) {
    let entity = hit.entity;
    let distance = hit.distance;
    let position = hit.world_position;
}
}

Pick all entities

#![allow(unused)]
fn main() {
let hits = pick_entities(world, screen_pos, PickingOptions::default());

for hit in &hits {
    let entity = hit.entity;
    let distance = hit.distance;
}
}

Picking options

#![allow(unused)]
fn main() {
pub struct PickingOptions {
    pub max_distance: f32,       // Maximum ray distance (default: infinity)
    pub ignore_invisible: bool,  // Skip entities with Visibility { visible: false } (default: true)
}
}

Triangle mesh picking

For pixel-precise picking, register entities for trimesh picking. The registration extracts the mesh vertices and indices from the entity's RenderMesh, applies the global transform's scale, and creates a SharedShape::trimesh collider positioned at the entity's world transform. The collider lives in the PickingWorld resource (a Rapier ColliderSet with entity-to-handle mappings).

#![allow(unused)]
fn main() {
use nightshade::ecs::picking::commands::*;

register_entity_for_trimesh_picking(world, entity);
}

For hierarchies (a parent with child meshes), one call walks the subtree.

#![allow(unused)]
fn main() {
register_entity_hierarchy_for_trimesh_picking(world, root_entity);
}

Trimesh raycasting

#![allow(unused)]
fn main() {
if let Some(hit) = pick_closest_entity_trimesh(world, screen_pos) {
    let entity = hit.entity;
    let distance = hit.distance;
    let position = hit.world_position;
}
}

The call casts a Rapier ray against every registered trimesh collider using shape.cast_ray(), returning the time of impact for each intersection.

Updating transforms

A pickable entity that moves needs its collider repositioned.

#![allow(unused)]
fn main() {
update_picking_transform(world, entity);
}

Unregistering

#![allow(unused)]
fn main() {
unregister_entity_from_picking(world, entity);
}

Pick result

#![allow(unused)]
fn main() {
pub struct PickingResult {
    pub entity: Entity,
    pub distance: f32,
    pub world_position: Vec3,
}
}

Utility functions

Ground plane intersection

The world position where a screen ray hits a horizontal plane.

#![allow(unused)]
fn main() {
if let Some(ground_pos) = get_ground_position_from_screen(world, screen_pos, 0.0) {
    // ground_pos is on the Y=0 plane
}
}

Frustum picking

Test which entities from a list are visible in the camera frustum. Each entity's bounding-sphere center is projected into clip space and tested against NDC bounds, accounting for the sphere radius in NDC space.

#![allow(unused)]
fn main() {
let visible = pick_entities_in_frustum(world, &entity_list);
}

Plane intersection

#![allow(unused)]
fn main() {
let ray = PickingRay::from_screen_position(world, screen_pos)?;

// Intersect with any plane (normal + distance from origin)
if let Some(point) = ray.intersect_plane(Vec3::y(), 0.0) {
    // point is on the plane
}

// Shorthand for horizontal ground plane
if let Some(point) = ray.intersect_ground_plane(0.0) {
    // point is on Y=0
}
}

Mouse click selection

#![allow(unused)]
fn main() {
fn on_mouse_input(&mut self, world: &mut World, state: ElementState, button: MouseButton) {
    if button == MouseButton::Left && state == ElementState::Pressed {
        let screen_pos = world.resources.input.mouse.position;

        if let Some(hit) = pick_closest_entity(world, screen_pos) {
            self.selected_entity = Some(hit.entity);
        } else {
            self.selected_entity = None;
        }
    }
}
}

Choosing between paths

Bounding volume picking is the right default. Its cost is bounded by the number of entities with bounding volumes, the math per entity is a sphere reject plus an OBB intersect, and there is no GPU dependency. It is fine for selection where the hit only needs to identify the entity, not the precise surface point.

Trimesh picking is the right call when the hit needs to be pixel-accurate against the rendered geometry. The cost is a Rapier raycast against every registered collider, so it scales with registration count rather than entity count. Entities have to be explicitly registered and their colliders updated when they move.

The GPU picking path (see the picking demo and the renderer source) is the right call when the renderer is already drawing the things to pick and the application can tolerate one frame of readback latency. It reuses the entity-id and depth textures the mesh pass already produces and gives world position plus surface normal in one readback. Lines, decals, and post-process effects are not drawn through the mesh shader, so the GPU path cannot pick them. The bounding-volume and trimesh paths cover those cases.