3rd Party OpenGL Instrumentation

From SIMboxWiki
Jump to navigation Jump to search

In the current technological environment there is a trend in replacing simple physical gauges with Monitors / MFDs (Multi functional display) gauges. In many of those cases the displays of these monitors are driven by programs rendering under OpenGL.

Up until now the standard modus operandi was to recreate the gauges appearance in SIMbox code. This means that code to render the more complex gauges needs to be created twice. Once for the actual physical console and once for the console in the simulation. This can waste resources.

To solve this issue the graphic engine offers a way to integrate 3rd party OpenGL code into the rendering process.

Prerequisites

The system requries the following prerequisites:

  • SIMbox version 5.6.1 or above
  • Window Vista or above
  • NVidia GeForce graphic card

System architecture

Wide view system architecture

The integration system was designed to support the following principals:

  1. System has to work with the graphic engine running either with OpenGL or Direct3D9; Direct3D9 being the more important of the two.
  2. The rendering of the 3rd party instrumentation should not influence the rendering process of the graphic engine.
  3. For optimal performance, the rendering process has to operate entirely within the medium of the graphic card. Rendering results cannot be copied to and fro between the graphic card memory and the main (motherboard) memory.
  4. System has to support the Side View architecture. This means that the system has be able to operate across multiple computers in parallel (Side View enables running the graphic engine on multiple computers to render multiple monitors).
  5. The system should support future extensions in which textures are supplied in other ways than OpenGL textures.

The architecture that was devised from these principals is thus. There are 3 components at play. A simulation Console Object Component (COC), the graphic engine and a new graphic engine plug-in known as Texture Source.

The COC is responsible for:

  1. Telling the graphic engine which texture source plug-ins (TSP) to load.
  2. Telling the texture source what type of gauges to create.
  3. Assigning the output of the texture source to one or more AGIs.
  4. Relaying information to the texture source. Information according to which the texture source should update its display.

The Texture Source plug-in (TSP) is responsible for:

  1. Creating gauges according to requests from the COC.
  2. Creating for each gauge an OpenGL texture to which to render the gauge to. Then handing over the texture to the graphic engine.
  3. Updating the gauge textures according to information sent from the COC.

The graphic engine is responsible for:

  1. Loading the texture source plug-ins.
  2. Relaying information between the COC and the Texture Source plug-ins.
  3. Using the textures handed from the Texture Source in the rendering process (as any other texture).

Texture Source plug-in architecture

The Texture Source plug-in (TSP) is a new SIMbox graphic engine plug-in. This plug-in is used to provide a way for 3rd party components to render their content into a texture. Then hand that texture to the SIMbox graphic engine to be used in the rendering process.

Consider the following scenario. Inside a cockpit of a multi-engine aircraft there are several gauges that need to be rendered from 3rd party components. Some gauges like air speed, glide-slope indicator and compass, are used once in a cockpit. While gauges such as engine speed, fuel flow and temperature indicators, have multiple instances, once per engine. The TSP needs to support creating several gauges. Where some of those gauges can be different instances of the same type of gauge.

To accommodate this situation the Texture Source was devised with the following architecture:

The TSP has 3 interfaces: SimTextureSource, SimTSInstance and SimTSOpenGLInstance.

  • SimTextureSource - This interface is used as the main interface between the graphic engine and the TSP. This interface is responsible creating the different gauges, relaying information between the COC and the individual gauges, and relaying requests for update between the graphic engine and the individual gauges.
  • SimTSInstance - This interface is used to a represent a single gauge. Each different gauge type should have its own implementation inheriting from this interface. Any logic regarding the rendering and updating of the gauge should be contained within that class.
  • SimTSOpenGLInstance - SimTSOpenGLInstance inherits from the SimTSInstance class. This class is used to represent gauges specifically rendered in OpenGL. The system is designed this way to allow for future developments in which textures are provided in by other means (i.e. raw data, video feed, etc...).

When a request to create a gauge is relayed from the COC, that request has 2 important parameters: A name and an instance number. The name should determine the type of gauge to create (air speed, compass, engine speed, fuel flow, etc ...). While the instance number should distinguish between different instances of the same gauge (e.g. fuel flow gauge for each of the engines in a multi-engine aircraft).

Once a gauge instance is created the API is devised as such that information relayed from the COC and graphic engine will target specific gauge instances.

Multiprocess

When using multiprocessed rendering the plugin DLL is loaded as part of the rendering process.
Note that from SIMbox version 5.7 the rendering process is 64 bit by default.
The plugin DLL will need to be compiled in 64 bit as well, the dlls and their dependencies should be placed under the bin\64 folder: C:\Program Files (x86)\KnowBook\Bin\64

API

Simulation side API

The simulation API for texture source is devised of the following functions. (For fully detailed information look in the file api.agi.h)

  • Agi::LoadTextureSourcePlugin() - Loads a Texture Source plug-in
  • Agi::CreateTextureSourceInstance() - Create a texture source instance. A renderable texture representing a gauge.
  • Agi::SetTextureSourceToAgi() - Sets a texture from a texture source instance onto an AGI
  • Agi::SendMessageToTextureSource - Sends a message buffer to a texture source instance. If operating under Side View mode, message is sent to all computers simultaneously.

TSP side API

The TSP API is devised of the following interfaces. (For fully detailed information look in the file api.SimTextureSource.h)

SimTextureSource

This interface provides the main contact point between the graphic engine and the TSP. One SimTextureSource is created be plug-in. A class inheriting from SimTextureSource should implement the following functions.

  • GenerateSourceInstance() - Called as a consequence of a component in the simulation calling CreateTextureSourceInstance(). If successful the function should return a pointer SimTSInstance representing a new texture source instance. If the TSP does not support generating texture sources of the requested type, the function should return a null pointer. In this eventuality the system will go to the next plug-in in line to request the texture source instance.
  • ReceiveData() - Called as a consequence of a component in the simulation calling SendMessageToTextureSource(). The function is called with the buffer that the user sent and a pointer to the SimTSInstance to which the data was directed at.
  • OnBeginFrame() - Called every time a new frame is about to be rendered. Certain Texture source implementation may require to update certain information on such event. Note that this does not necessarily mean that any of the texture source instances provided by the plug-in will be rendered. 
  • OnBeforeRenderInFrame() - Called every time a texture source instance is about to be rendered in a frame.

NOTE: You should only update your OpenGL texture during the calls GenerateSourceInstance() and OnBeforeRenderInFrame(). Doing so in any other function will produce unexpected results. For more information see the section on OpenGL context.

SimTSOpenGLInstance and SimTSInstance

The SimTSInstance interface represents a single texture source (equivalent to a single gauge). SimTSOpenGLInstance represents an implementation of SimTSInstance in which the texture is provided in the form of an OpenGL texture. For now all implementations of a texture source should inherit from SimTSOpenGLInstance.

There are 2 important methods to note here: SimTSInstance::Destroy() - This method may be called from the graphic engine to delete a SimTSInstance instance. This function comes pre-implemented. However if you wish to add extra code during the destruction phase of SimTSInstance this may be the place to do it.

SimTSOpenGLInstance::GetTextureId() - This method needs to be implemented to return a handle to the OpenGL texture on which the content of the texture source instance will be rendered. Note that this value must not change. It must be consistent throughout the life of the texture source.

TSP global functions

There is one global function in a TSP which needs to be implemented. This is the CreateSimTextureSource() function. This function is called when the plug-in is loaded to create a SimTextureSource instance for the graphic engine to interact with.

OpenGL context

Loading, manipulating and rendering of OpenGL resources should only be done under the function calls GenerateSourceInstance() and OnBeforeRenderInFrame(). The reason for this is the for each SimTSOpenGLInstance an opengl context is created by the graphic engine. However, because of performance considerations, this context is activated only during the calls to GenerateSourceInstance() and OnBeforeRenderInFrame(). Calling OpenGL API during other calls will mean that the code will not access the same resources.

Sample code

Description

This article comes with a sample file that provides an example for using the Texture Source interface.

The solution of the sample file contains 2 projects. The first GLTextureRenderDisplayComponent is a COC. Responsible for loading the TSP and instructing it what to display. The second GLTextureSourceSample is the TSP which generates the OpenGL textures.

The sample implements 2 type of gauges. The first type, called "SampleTexture", shows a simple implementation in which a bitmap is loaded as an OpenGL textures then displayed through the interface. The second type, called "SampleClock", shows a more complex gauge displaying a clock. This sample shows how to use OpenGL framebuffers to render complex content onto the displayed texture as well as handling messages between the COC and Source Texture plug-in.

The sample file can be downloaded here

Code walkthrough

The following text enumerates the main code sections in the sample code.

COC side code

We instruct the graphic engine to load the Texture Source plug-in

std::wstring dllPath = SimApi::Control::getBinPathW();
dllPath += L"GLTextureSourceSample.dll";
Agi::LoadTextureSourcePlugin(dllPath.c_str());

We create the AGIs on top of which the textures will be displayed

float width = 0, height = 0;
getElementSize(width,height);
m_hRectPlane = Agi::CreateRect(width / 2, height / 2, _T("Plane"));
m_hRectBuilding = Agi::CreateRect(width / 2, height / 2, _T("Building"));
m_hRectClock1 = Agi::CreateRect(width / 2, height / 2, _T("Clock1"));
m_hRectClock2 = Agi::CreateRect(width / 2, height / 2, _T("Clock2"));

Agi::SetXY(m_hRectPlane    ,0		  ,0);
Agi::SetXY(m_hRectBuilding ,width / 2 ,0);
Agi::SetXY(m_hRectClock1   ,0		  ,height / 2);
Agi::SetXY(m_hRectClock2   ,width / 2 ,height / 2);

We a create and assign sample textures to the AGIs we created. For "SampleTexture" type texture source this is done with the code:

//Initialization message for the gl texture 
//the message contains the path to the texture to load
MessageTextureInit initMessage;
wcscpy_s(initMessage.content.texturePath, 256, i_ImagePath.c_str());

//Create the texture
Agi::CreateTextureSourceInstance(L"SampleTexture", i_InstanceNumber, (const char *)&initMessage, sizeof(initMessage));
//Assign it to an AGI
Agi::SetTextureSourceToAgi(i_hAgi, L"SampleTexture", i_InstanceNumber);

For the "ClockSample" type texture source instance this is done with the code:

//Create a clock texture source
Agi::CreateTextureSourceInstance(L"SampleClock", i_InstanceNumber, NULL, 0);
//Assign a clock texture source to an AGI
Agi::SetTextureSourceToAgi(i_hAgi,L"SampleClock", i_InstanceNumber);

We also update the clock texture with the proper color scheme and behaviour we want by sending messages to it.

//send a message to update the behavior of the clock
MessageClockBehavior msgBehavior;
msgBehavior.content.isSmooth = reverseScheme;
Agi::SendMessageToTextureSource(L"SampleClock", i_InstanceNumber, (const char *)&msgBehavior, sizeof(msgBehavior));

//Send a message to updated the clocks color scheme
MessageClockColor colorMsg;
SetColorVec(colorMsg.content.backgroundColor, c_ColorTransparentDark);
SetColorVec(colorMsg.content.hourColor, reverseScheme ? c_ColorRed : c_ColorBlue);
SetColorVec(colorMsg.content.minuteColor, c_ColorGreen);
SetColorVec(colorMsg.content.secondColor, reverseScheme ? c_ColorBlue : c_ColorRed);
Agi::SendMessageToTextureSource(L"SampleClock", i_InstanceNumber, (const char *)&colorMsg, sizeof(colorMsg));

The last part is ensuring the the SampleClock type texture source instances are displying the correct time. We do this by sending them time update messages every time the COC's update function is called with the following code:

//update the time on both clocks
timeb tb;
ftime(&tb);

//Prepare the messate to update the time
MessageClockTime msgTime;
msgTime.content.hours = tb.time / 3600;
msgTime.content.minutes = (tb.time / 60) % 60;
msgTime.content.seconds = (tb.time) % 60;
msgTime.content.milliseconds = tb.millitm;

//send update time message to both instances of the clock
Agi::SendMessageToTextureSource(L"SampleClock", 0, (const char *)&msgTime, sizeof(msgTime));
Agi::SendMessageToTextureSource(L"SampleClock", 1, (const char *)&msgTime, sizeof(msgTime));

TSP side code

We start by setting up the TSP project environment. We create a simple dll to which we add the needed openGL header and libraries. We also add the TSP header "api.SimTextureSource.h" and define ourselves as exporters of the TSP methods by adding the preprocessor defitnion of SIMTEXTURESOURCE_DLL.

We then proceed to implement the TSP interface.

We need to ensure that the graphic engine can get the dll's inherited class of SimTextureSource. For this we implemented the DLL exproted function CreateSimTextureSource() thus.

SimTextureSource* CreateSimTextureSource()
{
	//Standard implementation for CreateSimTextureSource 
	//Simply create and return our implementation of SimTextureSource, SimTextureSourceSample.
	return new SimTextureSourceSample();
}

As discussed previously our TSP is able to produce 2 types of texture source instances. A texture source for simply displaying images saved on the disk and another for displying a clock. Each one of these is implemented in a different class. The classes inherit from an internal class called SimTSSampleInstance. Which itself inherits from the TSP interface SimTSOpenGLInstance.

We create instances of these classes depending on the user's request

SimTSInstance* SimTextureSourceSample::GenerateSourceInstace(const char* sourceName, unsigned int instanceId, const char* buffer, unsigned int bufferSize)
{
	SimTSSampleInstance* retInstance = NULL;
	//Check if we are requested to construct a SampleTexture type gauge
	if (_stricmp(sourceName, "SampleTexture") == 0)
	{
		retInstance = new SimTSSampleInstanceTextureLoad();
	}
	//Check if we are requested to construct a SampleClock type gauge
	else if (_stricmp(sourceName, "SampleClock") == 0)
	{
		retInstance = new SimTSSampleInstanceClock();
	}

	//If we have constructed a gauge initialize it
	if ((retInstance != NULL) && (retInstance->Init(buffer, bufferSize) == false))
	{
		//delete if there are errors
		delete retInstance;
		retInstance = NULL;
	}
	return retInstance;
}

We have created a texture handle parameter called m_TextureId in SimTSSampleInstance. To hand it to the graphic engine we implement the function GetTextureId() with the simple implementation of.

unsigned int SimTSSampleInstance::GetTextureId() const 
{ 
	return m_TextureId; 
}

Now each of the different gauges need to create the texture and assign content to it.

In the case of "TextureSample" we simply load the bitmap from the disk and display it. To load the file from the disk we use the glaux extention function auxDIBImageLoad. Then we create an openGL texture and load into it the content we loaded from the file thus.

// Load The Bitmap, Check For Errors, If Bitmap's Not Found Quit
if (textureImage = LoadBMP(filePath))
{
	//Create a texture on the graphic card
	glGenTextures(1, &m_TextureId);					

	//Assign the texture loaded to the texture in the graphic card
	glBindTexture(GL_TEXTURE_2D, m_TextureId);
	glTexImage2D(GL_TEXTURE_2D, 0, 3, textureImage->sizeX, textureImage->sizeY, 0, GL_RGB, GL_UNSIGNED_BYTE, textureImage->data);
	glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
	glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
}

The case of "ClockSample" is slighly more complex. We need to create a texture to which we can render our content. For this, along with creatina texture we create a framebuffer to which we can render our content and bind the framebuffer with the texture. We also create a secondary frame buffer to bind with the depthbuffer.

we generate a texture as before then

Generate a texture as before and assign it to m_TextureId

...

//Generate frame buffer
glGenFramebuffers(1, &m_FramebufferID); 
glGenRenderbuffers(1, &m_DepthRenderBufferID);  

...

//bind our frame buffer to the empty texture we created
glBindFramebuffer(GL_FRAMEBUFFER_EXT, m_FramebufferID);                       
glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT,GL_COLOR_ATTACHMENT0_EXT,GL_TEXTURE_2D, m_TextureId, 0); // 

// bind the depth frame buffer
// We don't really need a depth buffer for this sample but just for show
glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, m_DepthRenderBufferID);
// get the data space for it	
glRenderbufferStorageEXT(GL_RENDERBUFFER_EXT, GL_DEPTH_COMPONENT24, RTT_TEXTURE_WIDTH, RTT_TEXTURE_HEIGHT); 
// bind it to 
glFramebufferRenderbufferEXT(GL_FRAMEBUFFER_EXT,GL_DEPTH_ATTACHMENT_EXT,GL_RENDERBUFFER_EXT, m_DepthRenderBufferID); 

For the rendering cycle of the "ClockSample" we bind our RTT texture render our content flush the command pipeline and unbind the texture.

//bind the texture we are going to render into
glBindTexture(GL_TEXTURE_2D,m_TextureId);

...

Render the clock

...

//make sure our frame buffer / texture is fully updated
glFlush();	

//unbind our frame buffer
glBindFramebuffer(GL_FRAMEBUFFER_EXT, 0);  

Messaging

Much like transferring messages between network sockets, messages transfered between the COC and TSP take the form of simple naked buffers. This is meant to provide maximum performance and ease of use. In such cases, we have found basic template is helpfull for such message types to be created and handled.

You can view the file "SimTSMessages.h" in the provided sample for an example of such message structures.

We create simple message header

struct MessageHeader
{
	//The type of the message
	unsigned int messageId;
	//The size of the message
	unsigned int size;
};

We create a templated structure that will contain the message header with a structure of a message content (body)

template<typename MessageContent>
struct Message
{
	Message()
	{
		//Assign the message type and size
		header.messageId = MessageContent::MessageId;
		header.size = sizeof(MessageContent);
		//Roughly reset the message content
		memset(&content, 0, sizeof(MessageContent));
	}
	MessageHeader header;
	MessageContent content;
};

Notice that in our implementation of the templated Message structure we automatically update the header and reset the parameters of the content structure. This means that the content structure can only contain primitive types.

We create a message content. For example the message which updates the ClockSample gauge time

struct _MessageContentClockTime_
{
	enum { MessageId = 4 };
	//The number of indicated hours in the clock
	int hours;
	//The number of indicated minutes in the clock
	int minutes;
	//The number of indicated seconds in the clock
	int seconds;
	//The number of indicated milliseconds in the clock
	int milliseconds;
};

For ease of use, We bundle our time update content structure in a message with a header using the code.

typedef Message<_MessageContentClockTime_> MessageClockTime;

To send the message we add the code

timeb tb;
ftime(&tb);
MessageClockTime msgTime;
msgTime.content.hours = tb.time / 3600;
msgTime.content.minutes = (tb.time / 60) % 60;
msgTime.content.seconds = (tb.time) % 60;
msgTime.content.milliseconds = tb.millitm;
Agi::SendMessageToTextureSource(L"SampleClock", i_InstanceNumber, (const char *)&msgTime, sizeof(msgTime));

On the TSP side we create a switch case statement that will read the incoming message. This can be viewed in the function SimTSSampleInstanceClock::OnMessage.

void SimTSSampleInstanceClock::OnMessage(const char* i_Buffer, unsigned int i_BufferSize)
{
	//confirm we have a legal message header
	if ((i_Buffer != NULL) && (i_BufferSize >= sizeof(MessageHeader)))
	{
		//get the message size and type
		unsigned int messageId = ((const MessageHeader*)i_Buffer)->messageId;
		unsigned int messageSize = ((const MessageHeader*)i_Buffer)->size;
		//confirm we have a legal message again
		if (messageSize + sizeof(MessageHeader) <= i_BufferSize )
		{
			switch (messageId)
			{
			//Handle message for changing clock behavior
			case MessageClockBehavior::MessageId:
				{
					const MessageClockBehavior* msg = (const MessageClockBehavior*)i_Buffer;
					
					... Handle message MessageClockBehavior ...
				}
				break;
			//Handle message for changing color
			case MessageClockColor::MessageId:
				{
					const MessageClockColor* msg =  (const MessageClockColor*)i_Buffer;
					
					... Handle message MessageClockColor ...
				}
				break;
			//Handle the message which updates the time on the clock
			case MessageClockTime::MessageId:
				{
					const MessageClockTime* msg = (const MessageClockTime*)i_Buffer;
					
					... Handle message MessageClockTime ...
				}
				break;
			}
		}
	}
}

Exporting of the TSP's methods

When creating a Texture Source plug-in the preprocessor definition SIMTEXTURESOURCE_DLL must be included in the project's definition. Otherwise the functions and classes needed for the implementation of the Texture Source plug-in will not be exported

How to use

  1. Download the sample file and extract.
  2. Open the solution in visual studio.
  3. Compile the code in Release|Win32.
  4. Analyze the COC GLTextureRenderDisplayComponent.
  5. Compile the code again in Release|x64 (if using 64 bit multiprocess)
  6. Open the console builder.
  7. Add the GLTextureRenderDisplayComponent to a panel in one of your consoles