Image acquisition process
The configuration and image acquisition process of the depth camera are shown in the following figure:
Image acquisition flowchart
Below, we will use the SDK sample program Simpleview_FetchFrame as an example to explain the process of image acquisition.
Initialize API
TYInitLib initializes device objects and other data structures.
Open device
Get device list
Use selectDevice() to query the number of connected devices and get a list of all connected devices.
Open interface
TYOpenInterface opens interface.
Open device
TYOpenDevice opens the camera with the camera ID.
TYOpenDeviceWithIP opens the network camera with the camera IP.
Configure component
Check the component status of the device
TYGetComponentIDs gets information about the components supported by the device.
TYGetEnabledComponents gets information about the components that have been opened.
Configure component
TYEnableComponents enables components.
TYDisableComponents disables components.
After the device is opened, by default, only the virtual component TY_COMPONENT_DEVICE is enabled. Multiple components can be enabled simultaneously using the | operation.
int32_t componentIDs = TY_COMPONENT_DEPTH_CAM | TY_COMPONENT_RGB_CAM; TYEnableComponents(hDevice, componentIDs);
Configure features
Query information of specified features:
TYGetFeatureInfo() can get information about a specified feature of a specified component by filling the TY_FEATURE_INFO structure. If the specified component does not contain the specified feature, the isValid value in TY_FEATURE_INFO is false; if the component contains the specified feature, the isValid value in TY_FEATURE_INFO is true.
Alternatively, you can use specific API interfaces such as TYGetIntRange to query information about the specific feature.
The functions for reading and writing features are as follows:
TYGetIntRange TYGetInt TYSetInt TYGetFloatRange TYGetFloat TYSetFloat TYGetEnumEntryCount TYGetEnumEntryInfo TYGetEnum TYSetEnum TYGetBool TYSetBool TYGetStringLength TYGetString TYSetString TYGetStruct TYSetStruct
Sample program
Call TYSetEnum() to set the format and resolution of the depth image sensor output data:
LOGD("=== Configure feature, set resolution to 640x480."); ASSERT_OK(TYSetEnum(hDevice, TY_COMPONENT_DEPTH_CAM, TY_ENUM_IMAGE_MODE, TYImageMode2(TY_PIXEL_FORMAT_DEPTH16, 640, 480)));
Framebuffer management
Important
Before starting framebuffer management, make sure that the required components have been enabled using the TYEnableComponents() interface and that the image format and resolution have been set correctly using the TYSetEnum() interface. Because the size of the framebuffer depends on these settings, otherwise you may encounter problems with insufficient framebuffer space.
Call the TYGetFrameBufferSize() to query the size of each framebuffer under the current configuration.
uint32_t frameSize; ASSERT_OK( TYGetFrameBufferSize(hDevice, &frameSize) ); LOGD(" - Get size of framebuffer, %d", frameSize);
Allocate buffer for depth data.
Allocate two sets of frameBuffer space according to the actual size of the framebuffer returned by the query function, and pass them to the underlying driver buffer queue as channels for data acquisition.
The driver maintains an internal buffer queue (frameBuffer Queue). When each frame of data is transmitted, the filled frameBuffer is dequeued and fully transmitted for user use. After the user completes processing this frame of image data, an enqueue action must be performed to return the frameBuffer to the driver’s buffer queue. The user needs to ensure that the driver’s buffer queue is not full when a new frame of data arrives, otherwise the frame of data will be discarded.
LOGD(" - Allocate & enqueue buffers"); char* frameBuffer[2]; frameBuffer[0] = new char[frameSize]; frameBuffer[1] = new char[frameSize]; LOGD(" - Enqueue buffer (%p, %d)", frameBuffer[0], frameSize); ASSERT_OK( TYEnqueueBuffer(hDevice, frameBuffer[0], frameSize) ); LOGD(" - Enqueue buffer (%p, %d)", frameBuffer[1], frameSize); ASSERT_OK( TYEnqueueBuffer(hDevice, frameBuffer[1], frameSize) );
Callback function registration
TYRegisterEventCallback
The callback function for register event. The callback function registered by TYRegisterEventCallback is called when an exception event occurs. The callback function in the following example contains reconnected exception handling operations.
static bool offline = false;
void eventCallback(TY_EVENT_INFO *event_info, void *userdata)
{
if (event_info->eventId == TY_EVENT_DEVICE_OFFLINE) {
LOGD("=== Event Callback: Device Offline!");
// Note:
// Please set TY_BOOL_KEEP_ALIVE_ONOFF feature to false if you need to debug with breakpoint!
offline = true;
}
}
int main(int argc, char* argv[])
{
LOGD("Register event callback");
ASSERT_OK(TYRegisterEventCallback(hDevice, eventCallback, NULL))
while(!exit && !offline) {
//Fetch and process frame data
}
if (offline) {
//Release resources
TYStopCapture(hDevice);
TYCloseDevice(hDevice);
// Can try re-open and start device to capture image
// or just close interface exit
}
return 0;
}
Set work mode
Set the work mode of the depth camera according to actual needs. For more details, please refer to work mode settings.
Start capture
TYStartCapture
If the depth camera is working in mode 1, you can use the software trigger interface function TYSendSoftTrigger() to send commands via USB or Ethernet interface to control the timing of image acquisition.
Fetch frame
TYFetchFrame
In active depth data acquisition mode, the application can call this interface to retrieve depth data. After retrieving the data, the application should use an independent thread for processing to avoid blocking the image acquisition thread.
Stop capture
TYStopCapture stops image acquisition, and at the same time the camera stops calculating and outputting images.
Close device
TYCloseDevice closes the camera and TYCloseInterface frees up the occupied interfaces.
De-initialize API
After TYDeinitLib de-initializes the API, it is necessary to free up the allocated memory resources to avoid memory leaks.