00
*LIVE*
MoveNet
Left wrist[9]
Isolate a Single Point
The first technical experiment with BodyPose's MoveNet model. The experiment included isolating individual anchor points from the MoveNet keypoint array and isolating drawn connections. Tracked body points leave an organic, wobbly trace. Speed is already visible as an influencing element: the slower one moves, the more spikes and repetitive re-readings of the anchor point appear.
01
*LIVE*
MoveNet
Left wrist[9]
Right wrist[10]
Right hip[11]
Simultaneous Reconstructed Body
Using line as a primary element, three keypoints draw simultaneously, allowing for unexpected compositions and merging to occur. Different anchor points produce different speeds, range of coordinate positions, and distance from one another. Already, the body functions as a complex system and the interdependencies become obvious.
02
*LIVE*
MoveNet
Right wrist[10]
Right hip[11]
Pedestrian Distance Control
Isolating the interdependency of distance control (or lack of control). It’s interesting to see how an unconscious gesture such as the movement from walking leaves traces that can be used as design material. Movement tracking leaves an uneven line with variations based on small changes in location.
03
*LIVE*
MoveNet
Left wrist[9]
Visible Grids
Grids are typically something structured, static and stable. What does it look like to design or begin with grids that are constantly shifting and in motion? What further possibilities and new ideas emerge from unstable grids? The machinistic reduction to linear connections and keypoints led to an opposite exploration, where the body actively manipulates a grid.
04
Fill the video frame
and remain off-balance
MoveNet
Keypoints[i]
built off
BBBBB
ABCDe studio
OpenProcessing
Malleable Grids
Continued from Experiment 03, but introducing the entire body in grid manipulations. The grid distribution stems from the proportion of anchor points in each area.
05
MoveNet
Keypoints[i]
Fragmentation as Disguise
Responding to the fragmentation of a body into data points, and mirroring this fragmentation back to the way the data points are tracked. The typically structured tracking is broken across the screen by rotating and tiling the input video data. This is just one way to maybe, trick the machine, and bring back some ambiguity and agency into a standardized tracking process.
06
Staccato, sharp, precise movements
MoveNet
Left wrist[9]
Precise Typography?
If we move precisely, will the graphic render this way? Staccato, sharp movements with even time and spacing build the typographic composition.
07
Fluid, repetitive movement in a confined space
MoveNet
Left wrist[9]
Fluid Type Composition
What typographic compositions return from fluid, repetitive movement?
08
Fluid movement
MoveNet
Keypoints[i]
Organic Type Distortion
Typographic compositions that distort based on movement interaction.
09
*LIVE*
BlazePose
Right wrist[16]
Collecting Words, Forming Sentences
Fragments of a quote by Arturo Escobar, Designs for the Pluriverse.
10
[1]Angular, fast movement touching all levels in space
[3]Typical stretching sequence in modern dance
BlazePose
Keypoints[i]
Optical Flow
Motion history vectors
with
flow.js
kylemcdonald
Levels and Motion History
Crossing levels in space and mapping region data to shifting pixels in a letterform.
11
Free improv, flowing
BlazePose
Keypoints[i]
3D Keypoints
z value
Right hip[24]
Type Scrim
Free movement distorts a typographic composition, purposefully stretching legibility and shifting based on z-value. Text from André Lepecki's Exhuasting Dance, providing context for connections between choreography and type.
12
Free improv, flowing
BlazePose
3D Keypoints
z value
Right wrist[14]
with
p5variableFont.js
niraj94pangarkar
font [TINY]
Jack Halten Fahnestock
velvetyne.fr
Variable Type Scrim
Continued from Experiment 11, but mapping the weight axis to z value of the wrist.
13
Free improv, flowing
BlazePose
Keypoints
Nose[0]
Right shoulder[11]
Left shoulder[12]
Right hip[23]
Left hip[24]
with
p5variableFont.js
niraj94pangarkar
font [TINY]
Jack Halten Fahnestock
velvetyne.fr
In/visible Type
A formal experiment discovering possibilities of selectively manipulating a body of text. Intersecting bounds from each keypoint cause unexpected gradients of boldness.
14
Free improv, flowing
BlazePose
Keypoints
Nose[0]
Right wrist[15]
Left wrist[16]
Right hip[23]
Left hip[24]
Right foot[31]
Left foot[32]
font [TINY]
Jack Halten Fahnestock
velvetyne.fr
Moving Typewriter
How can we type with the entire body? Revealing only fragments of body text creates a different reading experience.
scroll▸
scroll▸
scroll▸
scroll▸
scroll▸
scroll▸
scroll▸
scroll▸
scroll▸
scroll▸
scroll▸
scroll▸
scroll▸