-
Notifications
You must be signed in to change notification settings - Fork 14
Added additional callbacks for arbitrary Data usage #250
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
added timestamp arg to sensor component added callback for dynamic data recorded
also added it to active session view
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds callback events for intercepting Cognitive3D data collection, enabling real-time machine learning applications and custom data processing. It includes a new "Data Connector" sample and updates multiple core recording systems to invoke callback events when data is collected or sent.
Key changes include:
- Addition of callback events for eye tracking, dynamic objects, sensors, and boundary data
- Breaking change:
onNewSensorRecordeddelegate signature updated to include timestamp parameter - New sample package for demonstrating data connector usage
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| package.json | Adds new "Data Connector" sample entry to the samples list |
| Runtime/Scripts/FixationRecorder.cs | Adds OnEyeDataRecorded event and EyeCaptureTimestampSeconds() methods across multiple eye tracking implementations |
| Runtime/Internal/Serialization/CoreInterface.cs | Invokes InvokeBoundarySend() when boundary data is posted |
| Runtime/Internal/SensorRecorder.cs | Updates onNewSensorRecorded delegate to include timestamp and invokes callback with timestamp |
| Runtime/Internal/EyeCapture.cs | Adds clarifying comment about Time field being unixtime in milliseconds |
| Runtime/Internal/DynamicManager.cs | Adds OnDynamicRecorded event and invokes it when dynamic objects are recorded |
| Runtime/Components/Boundary.cs | Adds OnBoundaryRecorded and OnBoundarySend events with invocation points |
| Runtime/ActiveSessionView/Scripts/SensorCanvas.cs | Updates sensor callback handler to accept new timestamp parameter |
| Runtime/ActiveSessionView/Scripts/DataBatchCanvas.cs | Adds boundary send tracking UI elements and event handlers |
| Runtime/ActiveSessionView/ActiveSessionView.prefab | Adds UI elements for boundary data tracking in ActiveSessionView |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Runtime/Internal/DynamicManager.cs
Outdated
| if (ActiveDynamicObjectsArray[i].active) | ||
| { | ||
| CoreInterface.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]); | ||
| InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale); |
Copilot
AI
Dec 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: Using ActiveInputsArray[i] while iterating over ActiveDynamicObjectsArray. This should be ActiveDynamicObjectsArray[i] to match the loop variable. This will cause an index out of bounds error when ActiveDynamicObjectsArray has more items than ActiveInputsArray, or will access incorrect data.
| InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale); | |
| InvokeDynamicRecorded(ActiveDynamicObjectsArray[i].Id, ActiveDynamicObjectsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveDynamicObjectsArray[i].LastPosition, ActiveDynamicObjectsArray[i].LastRotation, ActiveDynamicObjectsArray[i].LastScale); |
Runtime/Internal/DynamicManager.cs
Outdated
| InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale); | ||
| } | ||
| else | ||
| { | ||
| //DynamicObjectCore.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]); | ||
| CoreInterface.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]); | ||
| InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale); |
Copilot
AI
Dec 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: Using ActiveInputsArray[i] while iterating over ActiveDynamicObjectsArray. This should be ActiveDynamicObjectsArray[i] to match the loop variable. This will cause an index out of bounds error or access incorrect data.
| InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale); | |
| } | |
| else | |
| { | |
| //DynamicObjectCore.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]); | |
| CoreInterface.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]); | |
| InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale); | |
| InvokeDynamicRecorded(ActiveDynamicObjectsArray[i].Id, ActiveDynamicObjectsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveDynamicObjectsArray[i].LastPosition, ActiveDynamicObjectsArray[i].LastRotation, ActiveDynamicObjectsArray[i].LastScale); | |
| } | |
| else | |
| { | |
| //DynamicObjectCore.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]); | |
| CoreInterface.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]); | |
| InvokeDynamicRecorded(ActiveDynamicObjectsArray[i].Id, ActiveDynamicObjectsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveDynamicObjectsArray[i].LastPosition, ActiveDynamicObjectsArray[i].LastRotation, ActiveDynamicObjectsArray[i].LastScale); |
Runtime/Internal/DynamicManager.cs
Outdated
| InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale); | ||
| } | ||
| else | ||
| { | ||
| //DynamicObjectCore.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]); | ||
| CoreInterface.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]); | ||
| InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale); |
Copilot
AI
Dec 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: Using ActiveInputsArray[i] while iterating over ActiveDynamicObjectsArray. This should be ActiveDynamicObjectsArray[i] to match the loop variable. This will cause an index out of bounds error or access incorrect data.
| InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale); | |
| } | |
| else | |
| { | |
| //DynamicObjectCore.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]); | |
| CoreInterface.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]); | |
| InvokeDynamicRecorded(ActiveInputsArray[i].Id, ActiveInputsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveInputsArray[i].LastPosition, ActiveInputsArray[i].LastRotation, ActiveInputsArray[i].LastScale); | |
| InvokeDynamicRecorded(ActiveDynamicObjectsArray[i].Id, ActiveDynamicObjectsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveDynamicObjectsArray[i].LastPosition, ActiveDynamicObjectsArray[i].LastRotation, ActiveDynamicObjectsArray[i].LastScale); | |
| } | |
| else | |
| { | |
| //DynamicObjectCore.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]); | |
| CoreInterface.WriteDynamicManifestEntry(ActiveDynamicObjectsArray[i]); | |
| InvokeDynamicRecorded(ActiveDynamicObjectsArray[i].Id, ActiveDynamicObjectsArray[i].MeshName, Util.Timestamp(Time.frameCount), ActiveDynamicObjectsArray[i].LastPosition, ActiveDynamicObjectsArray[i].LastRotation, ActiveDynamicObjectsArray[i].LastScale); |
| var MsSincestart = currentData1.timestamp - startTimestamp; //milliseconds since start | ||
| var final = epochStart + MsSincestart; | ||
| return final; |
Copilot
AI
Dec 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: MsSincestart is in milliseconds (as indicated by the comment and variable name) but is being added directly to epochStart which is in seconds. This should be epochStart + (MsSincestart / 1000.0) to convert milliseconds to seconds before adding.
| var MsSincestart = currentData2.timestamp - startTimestamp; //milliseconds since start | ||
| var final = epochStart + MsSincestart; | ||
| return final; |
Copilot
AI
Dec 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: MsSincestart is in milliseconds (as indicated by the comment and variable name) but is being added directly to epochStart which is in seconds. This should be epochStart + (MsSincestart / 1000.0) to convert milliseconds to seconds before adding.
Runtime/Scripts/FixationRecorder.cs
Outdated
| //currentData.captureTime //nanoseconds. steady clock | ||
| long sinceStart = currentData.captureTime - startTimestamp; | ||
| sinceStart = (sinceStart / 1000000); //remove NANOSECONDS | ||
| var final = epochStart + sinceStart; |
Copilot
AI
Dec 10, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: After converting from nanoseconds to milliseconds at line 499, sinceStart is in milliseconds but is being added directly to epochStart which is in seconds. This should be epochStart + (sinceStart / 1000.0) to convert milliseconds to seconds before adding.
| var final = epochStart + sinceStart; | |
| var final = epochStart + (sinceStart / 1000.0); |
fixed fixation eye data timestamps
Description
Includes callbacks for alternative applications of collected Cognitive3D data, especially around realtime machine learning.
Type of change
Checklist