Skip to content

Conversation

@calderarchinuk
Copy link
Contributor

Description

Includes callbacks for alternative applications of collected Cognitive3D data, especially around realtime machine learning.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist

  • My changes will not interrupt or disrupt automated build processes
  • I have checked all outgoing links (i.e. to documentation) for validity
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published in downstream modules
  • I have assigned this PR to myself in GitHub
  • I have assigned and notified Reviewers for this PR
  • I have asked GitHub Copilot to review this PR

added timestamp arg to sensor component
added callback for dynamic data recorded
also added it to active session view
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds callback events for intercepting Cognitive3D data collection, enabling real-time machine learning applications and custom data processing. It includes a new "Data Connector" sample and updates multiple core recording systems to invoke callback events when data is collected or sent.

Key changes include:

  • Addition of callback events for eye tracking, dynamic objects, sensors, and boundary data
  • Breaking change: onNewSensorRecorded delegate signature updated to include timestamp parameter
  • New sample package for demonstrating data connector usage

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
package.json Adds new "Data Connector" sample entry to the samples list
Runtime/Scripts/FixationRecorder.cs Adds OnEyeDataRecorded event and EyeCaptureTimestampSeconds() methods across multiple eye tracking implementations
Runtime/Internal/Serialization/CoreInterface.cs Invokes InvokeBoundarySend() when boundary data is posted
Runtime/Internal/SensorRecorder.cs Updates onNewSensorRecorded delegate to include timestamp and invokes callback with timestamp
Runtime/Internal/EyeCapture.cs Adds clarifying comment about Time field being unixtime in milliseconds
Runtime/Internal/DynamicManager.cs Adds OnDynamicRecorded event and invokes it when dynamic objects are recorded
Runtime/Components/Boundary.cs Adds OnBoundaryRecorded and OnBoundarySend events with invocation points
Runtime/ActiveSessionView/Scripts/SensorCanvas.cs Updates sensor callback handler to accept new timestamp parameter
Runtime/ActiveSessionView/Scripts/DataBatchCanvas.cs Adds boundary send tracking UI elements and event handlers
Runtime/ActiveSessionView/ActiveSessionView.prefab Adds UI elements for boundary data tracking in ActiveSessionView

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

if (ActiveDynamicObjectsArray[i].active)
{
CoreInterface.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]);
InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale);
Copy link

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Using ActiveInputsArray[i] while iterating over ActiveDynamicObjectsArray. This should be ActiveDynamicObjectsArray[i] to match the loop variable. This will cause an index out of bounds error when ActiveDynamicObjectsArray has more items than ActiveInputsArray, or will access incorrect data.

Suggested change
InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale);
InvokeDynamicRecorded(ActiveDynamicObjectsArray[i].Id, ActiveDynamicObjectsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveDynamicObjectsArray[i].LastPosition, ActiveDynamicObjectsArray[i].LastRotation, ActiveDynamicObjectsArray[i].LastScale);

Copilot uses AI. Check for mistakes.
Comment on lines 1385 to 1391
InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale);
}
else
{
//DynamicObjectCore.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]);
CoreInterface.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]);
InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale);
Copy link

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Using ActiveInputsArray[i] while iterating over ActiveDynamicObjectsArray. This should be ActiveDynamicObjectsArray[i] to match the loop variable. This will cause an index out of bounds error or access incorrect data.

Suggested change
InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale);
}
else
{
//DynamicObjectCore.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]);
CoreInterface.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]);
InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale);
InvokeDynamicRecorded(ActiveDynamicObjectsArray[i].Id, ActiveDynamicObjectsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveDynamicObjectsArray[i].LastPosition, ActiveDynamicObjectsArray[i].LastRotation, ActiveDynamicObjectsArray[i].LastScale);
}
else
{
//DynamicObjectCore.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]);
CoreInterface.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]);
InvokeDynamicRecorded(ActiveDynamicObjectsArray[i].Id, ActiveDynamicObjectsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveDynamicObjectsArray[i].LastPosition, ActiveDynamicObjectsArray[i].LastRotation, ActiveDynamicObjectsArray[i].LastScale);

Copilot uses AI. Check for mistakes.
Comment on lines 1385 to 1391
InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale);
}
else
{
//DynamicObjectCore.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]);
CoreInterface.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]);
InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale);
Copy link

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Using ActiveInputsArray[i] while iterating over ActiveDynamicObjectsArray. This should be ActiveDynamicObjectsArray[i] to match the loop variable. This will cause an index out of bounds error or access incorrect data.

Suggested change
InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale);
}
else
{
//DynamicObjectCore.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]);
CoreInterface.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]);
InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale);
InvokeDynamicRecorded(ActiveDynamicObjectsArray[i].Id, ActiveDynamicObjectsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveDynamicObjectsArray[i].LastPosition, ActiveDynamicObjectsArray[i].LastRotation, ActiveDynamicObjectsArray[i].LastScale);
}
else
{
//DynamicObjectCore.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]);
CoreInterface.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]);
InvokeDynamicRecorded(ActiveDynamicObjectsArray[i].Id, ActiveDynamicObjectsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveDynamicObjectsArray[i].LastPosition, ActiveDynamicObjectsArray[i].LastRotation, ActiveDynamicObjectsArray[i].LastScale);

Copilot uses AI. Check for mistakes.
Comment on lines 380 to 382
var MsSincestart = currentData1.timestamp - startTimestamp; //milliseconds since start
var final = epochStart + MsSincestart;
return final;
Copy link

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: MsSincestart is in milliseconds (as indicated by the comment and variable name) but is being added directly to epochStart which is in seconds. This should be epochStart + (MsSincestart / 1000.0) to convert milliseconds to seconds before adding.

Copilot uses AI. Check for mistakes.
Comment on lines 388 to 390
var MsSincestart = currentData2.timestamp - startTimestamp; //milliseconds since start
var final = epochStart + MsSincestart;
return final;
Copy link

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: MsSincestart is in milliseconds (as indicated by the comment and variable name) but is being added directly to epochStart which is in seconds. This should be epochStart + (MsSincestart / 1000.0) to convert milliseconds to seconds before adding.

Copilot uses AI. Check for mistakes.
//currentData.captureTime //nanoseconds. steady clock
long sinceStart = currentData.captureTime - startTimestamp;
sinceStart = (sinceStart / 1000000); //remove NANOSECONDS
var final = epochStart + sinceStart;
Copy link

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: After converting from nanoseconds to milliseconds at line 499, sinceStart is in milliseconds but is being added directly to epochStart which is in seconds. This should be epochStart + (sinceStart / 1000.0) to convert milliseconds to seconds before adding.

Suggested change
var final = epochStart + sinceStart;
var final = epochStart + (sinceStart / 1000.0);

Copilot uses AI. Check for mistakes.
fixed fixation eye data timestamps
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants