Human pose and keypoint image datasets: visibility, coordinates and sample review

Plan still-image human pose datasets with joint definitions, occlusion states, 2D/3D distinctions, source review and an illustrative keypoint schema.

Human pose image datasets connect a person instance to defined body landmarks. They can support 2D localization, calibrated 3D reconstruction or broader body-model tasks, but those labels are not equivalent. A useful sourcing brief identifies the skeleton definition, coordinate frame, visibility policy, person grouping and review evidence. A convincing pose visualization alone does not establish measurement accuracy, participant permissions or suitability for a different model.

Primary source

Still photographs and their person-instance annotations

Critical distinction

Visible 2D landmark versus estimated or measured 3D location

Required metadata

Joint order, coordinate convention, visibility and person grouping

Example status

Custom teaching schema; no real participant record

Choose the pose target your model can actually use

Begin with the target: localizing visible body landmarks in a photograph is different from recovering metric body pose or fitting a parametric body model. A 2D wrist coordinate describes a location in image space. A 3D wrist coordinate requires a frame, units and a method for establishing depth. Neither implies a known contact force, a person’s intention or an executable robot action. Keep this page’s still-image buying decision separate from temporal action recognition and wearable motion capture.

The [COCO task index](https://cocodataset.org/) distinguishes keypoint estimation from detection and other tasks. The [CMU Panoptic toolbox](https://github.com/CMU-Perceptual-Computing-Lab/panoptic-toolbox) shows how synchronized views, calibration files and reconstructed skeletons can be delivered together. These references illustrate different supervision structures. They should not be ranked by a single image or joint count without examining their tasks, release versions and access conditions.

Write a skeleton definition, not just a list of joint names

Joint names are not enough. Specify the complete ordering, anatomical convention, left/right meaning and the location intended by each point. A shoulder can refer to a visible clothing contour, an anatomical approximation or a model-defined joint center. Labels with similar names may not be directly interchangeable. State whether left and right are the subject’s sides, and how mirrored images affect them. Preserve mappings when converting between schemas.

The [COCO reference API](https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/coco.py) and [Panoptic format documentation](https://github.com/CMU-Perceptual-Computing-Lab/panoptic-toolbox) provide concrete structures worth reading at the record level. A conversion should explicitly report unsupported landmarks rather than silently filling them with zeros. When combining archives, retain the native skeleton identifier as well as the normalized one. This allows a training or evaluation team to revisit a mapping error without trying to infer how the source originally defined its points.

Field

Why it matters

What to request

Skeleton/version

Joint order changes interpretation

Named ordered list and mapping version

Person instance

Multiple bodies can overlap

Stable within-image person identifier

Visibility state

Missing and occluded are different

Written state definitions

Coordinate frame

Pixels are not metric locations

Origin, axes, units and image transform

Source group

Related photos can leak across splits

Person/session/source relationships

Make occlusion and uncertainty explicit

An invisible wrist may still have a plausible estimated location, but plausibility is not direct evidence. Define separate states for visible, occluded but locatable, outside the image, not annotated and unknown. A project can choose a different state vocabulary, provided its meanings are explicit and preserved. Do not treat every absent keypoint as the same kind of missingness. A point beyond the crop and a point concealed behind an object create different learning and evaluation conditions.

Decide whether annotators may infer locations from body structure, use adjacent views or consult a reference frame. If so, record that evidence policy. Review difficult examples involving loose clothing, crossed limbs, wheelchairs, assistive devices, overlapping people and severe foreshortening with the actual schema. The goal is not to force a confident point on every body part. It is to preserve what the available evidence supports and mark the rest consistently.

Understand the difference between 2D and calibrated 3D

A collection described as “3D pose” needs more than three numbers per point. Ask which origin and axes are used, whether positions are camera-relative or world-relative, whether scale is metric, and whether root alignment has removed absolute position. Establish whether labels come from triangulation, a motion-capture system, a fitted model or a single-image predictor. These methods produce different uncertainty and failure patterns.

[OpenCV’s camera model documentation](https://docs.opencv.org/4.13.0/d9/d0c/group__calib3d.html) describes the relationship among camera intrinsics, extrinsics and projection. For a calibrated collection, request enough information to reproject a 3D label into the exact image delivered. A low average reprojection error is not a complete quality guarantee; it can hide localized failures or errors under unusual views. Review errors by camera, landmark, visibility and capture condition, and retain the calibration version used for the label.

Use a teaching example without inventing participant data

The diagram below is a drawn example rather than a photograph or a reconstructed person. It contains five named points to illustrate coordinate and state handling. It is deliberately not presented as COCO-compatible ground truth, a measured motion sequence or an available Fiund human-pose dataset. Its dashed wrist illustrates why an estimated occluded location should not be indistinguishable from a visible annotation.

[Download pose-keypoint-schema-example.json](/downloads/pose-keypoint-schema-example.json) includes image dimensions, a top-left pixel convention, an ordered point list and state labels. A production schema would need the complete target skeleton, reviewer rules and source records. The example can be used to test how a manifest stores uncertainty, but it should not be fed into a benchmark as though it were observed human data. This distinction protects both technical evaluation and the honesty of a public sample page.

Author-created coordinate diagram. No photograph, real participant, or measured pose is represented.

Plan coverage around body configuration and capture conditions

Useful coverage is multidimensional. Consider joint articulation, self-occlusion, camera elevation, lens distortion, framing, motion blur, clothing and interactions with objects. A dataset can include many people while rarely showing the body configurations a model must handle. Conversely, many poses of one person can leave important subject and appearance variation unexplored. Keep these dimensions visible instead of collapsing them into one broad diversity claim.

Ask for a coverage matrix with known and unknown fields. Do not infer sensitive participant attributes from appearance and treat them as verified demographic labels. If a project needs participant-reported information, define an appropriate collection and permission process. For deployment evaluation, select slices that reflect the actual operating setting and include difficult conditions intentionally, while avoiding the claim that a small convenience sample represents all human bodies or all geographical contexts.

Split by people and source relationships

A train/test split at the image level can leave nearly identical poses, bursts or repeated participants on both sides. Preserve session identifiers and known person relationships so a team can choose person-disjoint, session-disjoint or location-disjoint evaluation according to its objective. If identity relationships are unavailable, state that limit. Image hashes alone cannot establish that two different photographs depict different individuals.

Review how crops and augmentations inherit grouping. A horizontally flipped derivative, a close crop around a person and the full photograph should remain connected to their source. For multi-person scenes, explain whether each person crop becomes a separate training example and how the shared original is handled in evaluation. Count unique photographs, person instances and keypoint annotations separately so a buyer can understand what scale actually means.

Review an annotated sample as an engineering artifact

A pose pilot should provide the image, labels, schema and an overlay generated from those exact labels. Check image orientation, pixel bounds, left/right order and visibility meanings first. Then examine ambiguous anatomy and severe occlusion under the written policy. A beautiful hand-drawn overlay can conceal a broken export; generate the debugging view from the deliverable rather than from a separate illustration.

Choose acceptance criteria tied to the task. These may include missing-field limits, schema conformance, coordinate validity, measured localization quality on a reviewed subset and disagreement categories. Keep annotation mistakes separate from genuine uncertainty. Require correction records that identify which label changed and why. An acceptance report should name the reviewed sample and method, not make an unsupported assertion that an entire large archive was manually checked.

Quote the supervision, review and source scope separately

A commercial brief should specify accepted person instances or annotated images, the complete skeleton, the expected occlusion burden, any calibrated 3D requirement, participant/source documentation and the intended model use. Pose labels can require substantial review even when photographs already exist. A new capture project introduces different work: setup, participant recruitment, session control and retention of calibration or measurement evidence. These are distinct cost components.

No representative commercial transaction sample establishes a category-wide price in this guide. Ask suppliers to price the same accepted unit and correction responsibility. Confirm whether multiple people in one photograph count separately, whether ambiguous points are excluded or delivered with states, and what rework is included. Fiund can assess a sourcing specification and potential source fit; proposed availability, supporting documentation and commercial terms are confirmed for the selected material.

Is a 2D keypoint dataset suitable for 3D pose training?

It may contribute to a particular method, but it does not itself provide measured 3D joint positions. The training objective, camera information, target representation and required supervision must be established.

Should occluded points be removed?

That depends on the annotation policy. Preserve an explicit state so an estimated location is distinguishable from visible evidence, an out-of-frame point or a genuinely missing label.

Does this page provide a human-pose sample collection?

No. The diagram and JSON are illustrative schema resources. A real photographic collection requires separate source, participant, annotation and intended-use review.

Research method: Fiund research desk; public sources, explicit worked-example labels and project-specific availability review. Editorial contact: hello@fiund.com.

Sources
COCO task and dataset index: https://cocodataset.org/
COCO reference API and annotation structure: https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/coco.py
CMU Panoptic Studio official toolbox: https://github.com/CMU-Perceptual-Computing-Lab/panoptic-toolbox
OpenCV camera calibration and reconstruction documentation: https://docs.opencv.org/4.13.0/d9/d0c/group__calib3d.html