The objective of this work package is to develop the perceptual docking platform to enhance the surgeon’s interaction with the robot and provide effective control, guidance and visualization throughout the real or simulated interventional procedure. To meet this objective, the package is divided into the following aims:

• Provide a console based image guidance system with augmented reality visualization making use of real-time 3D information recovered optically from the operating field in conjunction with preoperative and intraoperative medical imaging data.
• Integrate perceptual docking on the master console by using eye tracking information to give surgeon a unique way of interacting with the robotic system and provide enhanced human-machine synergy.
• Integrate force and haptic feedback on the master controls of console to give the surgeon a sense of touch as well as enforce dynamic active constraints to protect delicate anatomical structures.

Where possible existing technology platforms will be used to meet these tasks. Hardware development is foreseen in: (1) haptics and force feedback device with sensing (2) eye tracking platform.  The remainder of the work package will be software focused particularly addressing the development of a video processing capability and the algorithms for meeting the outlined aims and the engineering of an integrated platform for simulation, planning and intraoperative assistance.