Chapter 17 — End-to-End Mobile Manipulator

17.1 Purpose and Scope

This chapter instantiates RelationWorld theory for a practical mobile manipulator architecture:

Mobile base: 4-wheel swerve drive on the plane
Manipulator: 4-DOF arm
Sensors: RGB/depth camera, 2D/3D LiDAR, IMU, odometry, joint states, optional force/contact

The aim is not to replace Chapters 1--16, but to provide an embodiment-level map:

{r a w se n sor} \to W_{t} \to r_{t} \to g_{t}^{{} {t a s k}} \to ξ_{t} \to u_{t} .

17.2 Ontological Role of the Robot

In this embodiment, the robot is specialised as:

mobile manipulator = stem traversal + door manipulation .

Hence, its operational priority is not arbitrary in-fruit dexterity, but reliable traversal over stem regions and intervention at door boundaries.

17.3 State and Action Model

17.3.1 Base state (swerve)

x_{t}^{b} = (x_{t}, y_{t}, θ_{t}), ν_{t} = (v_{{} x, t}, v_{{} y, t}, ω_{t}) .

Discrete-time kinematics (time step $Δ t$ ):

x_{t + 1} y_{t + 1} θ_{t + 1} = x_{t} + (v_{x, t} cos θ_{t} - v_{y, t} sin θ_{t}) Δ t, = y_{t} + (v_{x, t} sin θ_{t} + v_{y, t} cos θ_{t}) Δ t, = θ_{t} + ω_{t} Δ t .

17.3.2 Arm state

q_{t}^{a} = (q_{{} 1, t}, q_{{} 2, t}, q_{{} 3, t}, q_{{} 4, t}), \overset{q}{˙}_{t}^{a} = (\overset{q}{˙}_{{} 1, t}, \overset{q}{˙}_{{} 2, t}, \overset{q}{˙}_{{} 3, t}, \overset{q}{˙}_{{} 4, t}) .

17.3.3 Combined robot state and action

x_{t}^{r} = (x_{t}^{b}, q_{t}^{a}, ν_{t}, \overset{q}{˙}_{t}^{a}) .

u_{t}^{{} {mb}} = (v_{{} x, t}, v_{{} y, t}, ω_{t}, \overset{q}{˙}_{{} 1, t}, \overset{q}{˙}_{{} 2, t}, \overset{q}{˙}_{{} 3, t}, \overset{q}{˙}_{{} 4, t}) .

17.4 Base-Arm Coupled Kinematics

Let end-effector pose be

p_{t}^{e} = f_{FK} (x_{t}^{b}, q_{t}^{a}) .

Then velocity is

\overset{p}{˙}_{t}^{e} = J_{b} (x_{t}^{b}, q_{t}^{a}) ν_{t} + J_{a} (x_{t}^{b}, q_{t}^{a}) \overset{q}{˙}_{t}^{a},

or compactly

\overset{p}{˙}_{t}^{e} = J_{whole} (x_{t}^{r}) ν_{t}^{mb} .

This coupling equation is the mechanical core of the end-to-end formulation.

17.5 Sensor to Relational Field

Sensor stream:

y_{t} = (I_{t}, D_{t}, L_{t}, {o d o m}_{t}, q_{t}^{a}, \overset{q}{˙}_{t}^{a}, {f or ce}_{t}, \dots) .

History encoder:

z_{t} = E_{ϕ} (y_{0 : t}) .

Node/edge construction:

V_{t} = N_{ϕ} (z_{t}), r_{t} (i, j) = (i, j, w_{t} (i, j), g_{t} (i, j)) .

Embodiment-conditioned edge score (example):

w_{t} (i, j) = α_{1} proximity_{ij} + α_{2} handover_{ij} + α_{3} cofunc_{ij} + α_{4} causal_{ij} + α_{5} manipulability_{ij} .

17.6 World Decomposition

Given thresholds $θ \in (0, 1)$ and $τ > 0$ :

F {i s f r u i t} ⟺ ϕ (F) = \frac{{}{∣} \partial F ∣_{w}} {{v o l} (F)} \leq θ,

S_{t} = V_{t} ∖ k ⋃ F_{k},

Σ (F) = {i \in F : b_{F} (i) \geq τ}, b_{F} (i) = d (i) - d_{F}^{{} {in t}} (i) .

World summary:

W_{t} = ({F_{k}}, S_{t}, {Σ (F_{k})}) .

17.7 Role Inference and Task Generation

Role family:

R = {{s t e m - t r an s p or t er}, {d oor - o p er a t or}, {d oor - s t abi l i z er}} .

Role selection:

r_{t} = ar g r \in R max [β_{1} Δ Flow (r) + β_{2} Δ BoundaryStability (r) + β_{3} Δ ManipulationUtility (r) - β_{4} Cost (r) - β_{5} Risk (r)] .

World deficit:

Δ_{t} = λ_{1} {S t e m C o n g es t i o n} + λ_{2} {D oor O v er l o a d} + λ_{3} {B u f f er M i s ma t c h} + λ_{4} {H u man B u r d e n} + λ_{5} {I d l e L oss} .

Task generation:

g_{t}^{task} = π_{task} (r_{t}, W_{t}, Δ_{t}) .

Each task is decomposed as

g_{t}^{{} {t a s k}} \mapsto (g_{t}^{{} {na v}}, g_{t}^{{} {mani p}}),

with canonical execution phases:

approach \to align \to manipulate \to exit .

For the first benchmark family (door handle/tray), these phases are mandatory.

17.9 Motion Latent and Hybrid End-to-End Objective

Motion latent:

ξ_{t} = π_{{} {m o t i o n}} (y_{{} 0 : t}, W_{t}, r_{t}, g_{t}^{{} {t a s k}}), ξ_{t} = (ξ_{t}^{b}, ξ_{t}^{a}) .

Coupled objective (horizon $H$ ):

x^{b}, q^{a}, u min τ = t \sum t + H (ℓ_{task} + λ_{1} ℓ_{world} + λ_{2} ℓ_{reach} + λ_{3} ℓ_{safety} + λ_{4} ℓ_{energy}) .

where

ℓ_{{} {t a s k}} = ∥ f_{{} {F K}} (x_{τ}^{b}, q_{τ}^{a}) - p_{τ}^{⋆} ∥^{2},

ℓ_{world} = γ_{1} dist (x_{τ}^{b}, C_{stem})^{2} + γ_{2} door_alignment (x_{τ}^{b}, Σ) + γ_{3} boundary_respect,

ℓ_{{} {r e a c h}} = {I K_r es i d u a l} (x_{τ}^{b}, q_{τ}^{a}, p_{τ}^{⋆})^{2} .

17.10 ROS 2 Interface Contract

Implementation-facing message/action contracts are standardised in package antbot_relational_e2e:

RelationalWorld.msg
RoleAssignment.msg
GeneratedTask.msg
DoorManip.action

This contract is the bridge between theory-level variables and executable ROS graphs.

17.11 Traceability to Core Theory

Core theory (Ch.1--16)	Embodiment mapping (this chapter)
Relation tuple and weighted field	Sensor-conditioned graph features
Fruit/stem/door decomposition	Workspace partition and door intervention points
Theorems A--H (stability and structure)	Constraints and penalties in world-consistency objective
Existence and world reading	Role assignment + task generation pipeline
Dynamics (Ch.14)	Online update of world graph and role/task refresh

17.12 Validation Gates

To move from specification to simulation:

Dimensional consistency of state/action/kinematics equations.
Interface completeness for world, role, task, and manipulation action.
Task closure: door-handle/tray scenario must complete all four phases.
Objective traceability: each objective term mapped to at least one theoretical primitive.

These gates define readiness for the simulation implementation phase.

17.1 Purpose and Scope

This chapter instantiates RelationWorld theory for a practical mobile manipulator architecture:

Mobile base: 4-wheel swerve drive on the plane
Manipulator: 4-DOF arm
Sensors: RGB/depth camera, 2D/3D LiDAR, IMU, odometry, joint states, optional force/contact

The aim is not to replace Chapters 1--16, but to provide an embodiment-level map:

{r a w se n sor} \to W_{t} \to r_{t} \to g_{t}^{{} {t a s k}} \to ξ_{t} \to u_{t} .

17.2 Ontological Role of the Robot

In this embodiment, the robot is specialised as:

mobile manipulator = stem traversal + door manipulation .

Hence, its operational priority is not arbitrary in-fruit dexterity, but reliable traversal over stem regions and intervention at door boundaries.

17.3 State and Action Model

17.3.1 Base state (swerve)

x_{t}^{b} = (x_{t}, y_{t}, θ_{t}), ν_{t} = (v_{{} x, t}, v_{{} y, t}, ω_{t}) .

Discrete-time kinematics (time step $Δ t$ ):

x_{t + 1} y_{t + 1} θ_{t + 1} = x_{t} + (v_{x, t} cos θ_{t} - v_{y, t} sin θ_{t}) Δ t, = y_{t} + (v_{x, t} sin θ_{t} + v_{y, t} cos θ_{t}) Δ t, = θ_{t} + ω_{t} Δ t .

17.3.2 Arm state

q_{t}^{a} = (q_{{} 1, t}, q_{{} 2, t}, q_{{} 3, t}, q_{{} 4, t}), \overset{q}{˙}_{t}^{a} = (\overset{q}{˙}_{{} 1, t}, \overset{q}{˙}_{{} 2, t}, \overset{q}{˙}_{{} 3, t}, \overset{q}{˙}_{{} 4, t}) .

17.3.3 Combined robot state and action

x_{t}^{r} = (x_{t}^{b}, q_{t}^{a}, ν_{t}, \overset{q}{˙}_{t}^{a}) .

u_{t}^{{} {mb}} = (v_{{} x, t}, v_{{} y, t}, ω_{t}, \overset{q}{˙}_{{} 1, t}, \overset{q}{˙}_{{} 2, t}, \overset{q}{˙}_{{} 3, t}, \overset{q}{˙}_{{} 4, t}) .

17.4 Base-Arm Coupled Kinematics

Let end-effector pose be

p_{t}^{e} = f_{FK} (x_{t}^{b}, q_{t}^{a}) .

Then velocity is

\overset{p}{˙}_{t}^{e} = J_{b} (x_{t}^{b}, q_{t}^{a}) ν_{t} + J_{a} (x_{t}^{b}, q_{t}^{a}) \overset{q}{˙}_{t}^{a},

or compactly

\overset{p}{˙}_{t}^{e} = J_{whole} (x_{t}^{r}) ν_{t}^{mb} .

This coupling equation is the mechanical core of the end-to-end formulation.

17.5 Sensor to Relational Field

Sensor stream:

y_{t} = (I_{t}, D_{t}, L_{t}, {o d o m}_{t}, q_{t}^{a}, \overset{q}{˙}_{t}^{a}, {f or ce}_{t}, \dots) .

History encoder:

z_{t} = E_{ϕ} (y_{0 : t}) .

Node/edge construction:

V_{t} = N_{ϕ} (z_{t}), r_{t} (i, j) = (i, j, w_{t} (i, j), g_{t} (i, j)) .

Embodiment-conditioned edge score (example):

w_{t} (i, j) = α_{1} proximity_{ij} + α_{2} handover_{ij} + α_{3} cofunc_{ij} + α_{4} causal_{ij} + α_{5} manipulability_{ij} .

17.6 World Decomposition

Given thresholds $θ \in (0, 1)$ and $τ > 0$ :

F {i s f r u i t} ⟺ ϕ (F) = \frac{{}{∣} \partial F ∣_{w}} {{v o l} (F)} \leq θ,

S_{t} = V_{t} ∖ k ⋃ F_{k},

Σ (F) = {i \in F : b_{F} (i) \geq τ}, b_{F} (i) = d (i) - d_{F}^{{} {in t}} (i) .

World summary:

W_{t} = ({F_{k}}, S_{t}, {Σ (F_{k})}) .

17.7 Role Inference and Task Generation

Role family:

R = {{s t e m - t r an s p or t er}, {d oor - o p er a t or}, {d oor - s t abi l i z er}} .

Role selection:

r_{t} = ar g r \in R max [β_{1} Δ Flow (r) + β_{2} Δ BoundaryStability (r) + β_{3} Δ ManipulationUtility (r) - β_{4} Cost (r) - β_{5} Risk (r)] .

World deficit:

Δ_{t} = λ_{1} {S t e m C o n g es t i o n} + λ_{2} {D oor O v er l o a d} + λ_{3} {B u f f er M i s ma t c h} + λ_{4} {H u man B u r d e n} + λ_{5} {I d l e L oss} .

Task generation:

g_{t}^{task} = π_{task} (r_{t}, W_{t}, Δ_{t}) .

Each task is decomposed as

g_{t}^{{} {t a s k}} \mapsto (g_{t}^{{} {na v}}, g_{t}^{{} {mani p}}),

with canonical execution phases:

approach \to align \to manipulate \to exit .

For the first benchmark family (door handle/tray), these phases are mandatory.

17.9 Motion Latent and Hybrid End-to-End Objective

Motion latent:

ξ_{t} = π_{{} {m o t i o n}} (y_{{} 0 : t}, W_{t}, r_{t}, g_{t}^{{} {t a s k}}), ξ_{t} = (ξ_{t}^{b}, ξ_{t}^{a}) .

Coupled objective (horizon $H$ ):

x^{b}, q^{a}, u min τ = t \sum t + H (ℓ_{task} + λ_{1} ℓ_{world} + λ_{2} ℓ_{reach} + λ_{3} ℓ_{safety} + λ_{4} ℓ_{energy}) .

where

ℓ_{{} {t a s k}} = ∥ f_{{} {F K}} (x_{τ}^{b}, q_{τ}^{a}) - p_{τ}^{⋆} ∥^{2},

ℓ_{world} = γ_{1} dist (x_{τ}^{b}, C_{stem})^{2} + γ_{2} door_alignment (x_{τ}^{b}, Σ) + γ_{3} boundary_respect,

ℓ_{{} {r e a c h}} = {I K_r es i d u a l} (x_{τ}^{b}, q_{τ}^{a}, p_{τ}^{⋆})^{2} .

17.10 ROS 2 Interface Contract

Implementation-facing message/action contracts are standardised in package antbot_relational_e2e:

RelationalWorld.msg
RoleAssignment.msg
GeneratedTask.msg
DoorManip.action

This contract is the bridge between theory-level variables and executable ROS graphs.

17.11 Traceability to Core Theory

Core theory (Ch.1--16)	Embodiment mapping (this chapter)
Relation tuple and weighted field	Sensor-conditioned graph features
Fruit/stem/door decomposition	Workspace partition and door intervention points
Theorems A--H (stability and structure)	Constraints and penalties in world-consistency objective
Existence and world reading	Role assignment + task generation pipeline
Dynamics (Ch.14)	Online update of world graph and role/task refresh

17.12 Validation Gates

To move from specification to simulation:

Dimensional consistency of state/action/kinematics equations.
Interface completeness for world, role, task, and manipulation action.
Task closure: door-handle/tray scenario must complete all four phases.
Objective traceability: each objective term mapped to at least one theoretical primitive.

These gates define readiness for the simulation implementation phase.

Chapter 17 — End-to-End Mobile Manipulator

17.1 Purpose and Scope

17.2 Ontological Role of the Robot

17.3 State and Action Model

17.3.1 Base state (swerve)

17.3.2 Arm state

17.3.3 Combined robot state and action

17.4 Base-Arm Coupled Kinematics

17.5 Sensor to Relational Field

17.6 World Decomposition

17.7 Role Inference and Task Generation

17.8 Navigation-Manipulation Pairing

17.9 Motion Latent and Hybrid End-to-End Objective

17.10 ROS 2 Interface Contract

17.11 Traceability to Core Theory

17.12 Validation Gates

Chapter 17 — End-to-End Mobile Manipulator

17.1 Purpose and Scope

17.2 Ontological Role of the Robot

17.3 State and Action Model

17.3.1 Base state (swerve)

17.3.2 Arm state

17.3.3 Combined robot state and action

17.4 Base-Arm Coupled Kinematics

17.5 Sensor to Relational Field

17.6 World Decomposition

17.7 Role Inference and Task Generation

17.8 Navigation-Manipulation Pairing

17.9 Motion Latent and Hybrid End-to-End Objective

17.10 ROS 2 Interface Contract

17.11 Traceability to Core Theory

17.12 Validation Gates

17.1 Purpose and Scope#

17.2 Ontological Role of the Robot#

17.3 State and Action Model#

17.3.1 Base state (swerve)#

17.3.2 Arm state#

17.3.3 Combined robot state and action#

17.4 Base-Arm Coupled Kinematics#

17.5 Sensor to Relational Field#

17.6 World Decomposition#

17.7 Role Inference and Task Generation#

17.8 Navigation-Manipulation Pairing#

17.9 Motion Latent and Hybrid End-to-End Objective#

17.10 ROS 2 Interface Contract#

17.11 Traceability to Core Theory#

17.12 Validation Gates#

17.1 Purpose and Scope#

17.2 Ontological Role of the Robot#

17.3 State and Action Model#

17.3.1 Base state (swerve)#

17.3.2 Arm state#

17.3.3 Combined robot state and action#

17.4 Base-Arm Coupled Kinematics#

17.5 Sensor to Relational Field#

17.6 World Decomposition#

17.7 Role Inference and Task Generation#

17.8 Navigation-Manipulation Pairing#

17.9 Motion Latent and Hybrid End-to-End Objective#

17.10 ROS 2 Interface Contract#

17.11 Traceability to Core Theory#

17.12 Validation Gates#

17.1 Purpose and Scope

17.2 Ontological Role of the Robot

17.3 State and Action Model

17.3.1 Base state (swerve)

17.3.2 Arm state

17.3.3 Combined robot state and action

17.4 Base-Arm Coupled Kinematics

17.5 Sensor to Relational Field

17.6 World Decomposition

17.7 Role Inference and Task Generation

17.8 Navigation-Manipulation Pairing

17.9 Motion Latent and Hybrid End-to-End Objective

17.10 ROS 2 Interface Contract

17.11 Traceability to Core Theory

17.12 Validation Gates

17.1 Purpose and Scope

17.2 Ontological Role of the Robot

17.3 State and Action Model

17.3.1 Base state (swerve)

17.3.2 Arm state

17.3.3 Combined robot state and action

17.4 Base-Arm Coupled Kinematics

17.5 Sensor to Relational Field

17.6 World Decomposition

17.7 Role Inference and Task Generation

17.8 Navigation-Manipulation Pairing

17.9 Motion Latent and Hybrid End-to-End Objective

17.10 ROS 2 Interface Contract

17.11 Traceability to Core Theory

17.12 Validation Gates