Wireles Interactive Networked Display (WIND)

This document specifies the Wireles Interactive Networked Display (WIND) device. It covers the device architecture, the components and their state machines, the interfaces between them, the handlers wrapping external resources, and the messages and functions used throughout. Cross-references are clickable; hover the underlined identifiers to navigate to their definitions.

Device specification

The device is a Wireless Interactive Networked Display (WIND) which is controlled by a central server (not in scope). It operates by creating a connection to the server and then handling requests to display content formatted as a PNG image. Some content must be interactive which is enabled by the display being touch sensitive and the server sending an indexed coordinates map of touch-squares on the display. The device must be extremely low cost, so interaction is limited by the device mapping touch coordinates to the proper id and sending that id back to the server.

The device further has a speaker for supplying audio cues, which comprises a set of indexed predefined sounds. Each square on the touch-map can have an audio cue associated with it, specified by the index of the sound. The server can update the stored audio samples.

Device architecture

The descriptions of the identified components are given in the table below.

Component	Description
Server Connector	Controls and establishes the connection with the server. It waits for the device to become ready and then starts advertising its presence on the network to be discoverable by a server.
Device Status Controller	Takes care of initializing the components in the device by waiting for all the peripherals to be operational.
UI Controller	Controls the user interaction by interfacing with the TouchDisplay and the AudioHandler.

The descriptions of the identified handlers are given in the table below.

Handler	Description
Wireless Transceiver	This handler takes care of receiving and transmitting data packets on the radio front-end by interfacing with the radio hardware peripheral. By looking at the message-type of incoming messages, indications are forwarded either to the Connection or Content interface.
Peripheral Handler	Takes care of detecting, initializing and configuring hardware peripherals, e.g. by accessing low level drivers and serial communication lines.
Touch Display	Handles the interface to the touch display to show images and capture touch events.
Audio Handler	Enables the ability to play audio cues to the user upon touch events. It stores audio samples internally, which can be updated via the interface.

Interfaces

Interfaces define the message vocabulary that connects components and handlers.

The following table lists the defined interfaces.

Name	Description
Audio	Status indication from the audio.
Connection	Wireless transceiver messages for connection handling.
Content	Exchange of interaction with the server.
Device	Device status messages.
Display	Display status and control messages.
Peripherals	Status indications of all the peripherals.

The messages for the interfaces are specified in the following sections.

Device

Device status messages.

Message	Message description	Parameter	Type	Parameter description
ErrorInd	An error occurred and the device is no longer operational.	—	—	—
ReadyInd	All peripherals are ready and the device can be initialised.	—	—	—

Connection

Wireless transceiver messages for connection handling.

Message	Message description	Parameter	Type	Parameter description
AdvertiseInd	Broadcast to be discoverable.	id	uint	ID of the device.
ConnectedInd	Connected to the server.	—	—	—
ConnectReq	Request received to connect to a server.	serverId	uint	ID of the server what wants to connect.
DisconnectedInd	The transceiver detected that the server is disconnected.	—	—	—
ReadyInd	The transceiver is ready for operation.	—	—	—

Peripherals

Status indications of all the peripherals.

Message	Message description	Parameter	Type	Parameter description
FailureInd	A peripheral initialisation failed or is no longer operational.	—	—	—
ReadyInd	Indication that a specific peripheral is ready.	peripheralId	uint	ID of the peripheral that became operational.

Content

Exchange of interaction with the server.

Message	Message description	Parameter	Type	Parameter description
ContentInd	A new image to display with touch map and audio queues.	image	byteMatrix	The PNG image to display.
		soundMap	uintArray	Maps a touch ID to a sound ID.
		touchMap	uintMatrix	Mapping the pixels in the image to touch area IDs.
SoundInd	An update of a sound.	id	uint	The ID of the location where to place the sound.
SoundInd	An update of a sound.	soundData	byteArray	The audio samples.

Audio

Status indication from the audio.

Message	Message description	Parameter	Type	Parameter description
ReadyInd	Indication that the audio device is ready.	numSounds	uint	The number of stored sounds in the audio device.

Display

Display status and control messages.

Message	Message description	Parameter	Type	Parameter description
ReadyInd	Indication that the touch display is initialized.	height	uint	The height of the display in pixels.
ReadyInd	Indication that the touch display is initialized.	width	uint	The width of the display in pixels.
TouchedInd	Indication that a touch event occurred on the display.	x	uint	The x-coordinate of the touched pixel.
TouchedInd	Indication that a touch event occurred on the display.	y	uint	The y-coordinate of the touched pixel.

Server Connector

Controls and establishes the connection with the server. It waits for the device to become ready and then starts advertising its presence on the network to be discoverable by a server.

Context

Constants

Name	Value	Description
ADVERTISEMENT	1000	Advertisement interval in ms.
CONNECTION_TIMEOUT	500	Timeout waiting for server to connect in ms.

State variables

Name	Type	Description
serverId	uint	The ID of the connected or connecting server.
whitelist	uintArray	Whitelist of servers.

States

State	Description
Advertising	Broadcasting advertisement packets on the network to be discoverable by a server.
Connected	Connected to the server.
Connecting	Establishing a connection to the server.
Initialising	Waiting until all components are ready.
Uninitialised	Waiting for the device to become ready.

Choice-points

Question	Description
Is Server Whitelisted?	Check whether the server that tries to connect is whitelisted.

State diagram

State transition table

Source	Target	Interface	Message	Action
●	Uninitialised	—	—	—
Advertising	Is Server Whitelisted?	Connection	ConnectReq	Stop the timer. serverId =ConnectReq:serverId
Advertising	Advertising	Timer	Timeout	Call doAdvertise().
Connected	Connected	Connection	ConnectedInd	—
Connected	Advertising	Connection	DisconnectedInd	Call doAdvertise().
Connecting	Connected	Connection	ConnectedInd	—
Connecting	Advertising	Timer	Timeout	Call doAdvertise().
Initialising	Advertising	Connection	ReadyInd	Call doAdvertise().
Is Server Whitelisted?	Advertising	Logical	No	Call the WirelessTransceiver:rejectServer(). Call the WirelessTransceiver:sendAdvertisementPacket(). Start the timer with the ADVERTISEMENT interval.
Is Server Whitelisted?	Connecting	Logical	Yes	Call WirelessTransceiver:connectServer(serverId). Start the timer with the CONNECTION_TIMEOUT interval.
Uninitialised	Uninitialised	*	*	—
Uninitialised	Initialising	Device	ReadyInd	—
∗	Uninitialised	Device	ErrorInd	Call the WirelessTransceiver:disable().

Local functions

Name	Description	Steps
doAdvertise	Send advertisement packet and start timer.	Call the WirelessTransceiver:sendAdvertisementPacket(). Start the timer with the ADVERTISEMENT interval.

Device Status Controller

Takes care of initializing the components in the device by waiting for all the peripherals to be operational.

Context

Constants

Name	Value	Description
NUM_PERIPHERALS	3	Number of expected peripherals.

State variables

Name	Type	Description
peripherals	uintArray	List of operational peripherals.

States

State	Description
Awaiting Peripherals	Waiting for all the peripherals to become ready.
Error	Initialisation of the device failed or peripherals became unavailable.
Operational	All peripherals are operational.

Choice-points

Question	Description
All Peripherals Operational?	Are all expected NUM_PERIPHERALS peripherals operational?

State diagram

State transition table

Source	Target	Interface	Message	Action
●	Awaiting Peripherals	—	—	—
All Peripherals Operational?	Awaiting Peripherals	Logical	No	—
All Peripherals Operational?	Operational	Logical	Yes	Send the Device:ReadyInd message.
Awaiting Peripherals	Error	Peripherals	FailureInd	Send the Device:ErrorInd message.
Awaiting Peripherals	All Peripherals Operational?	Peripherals	ReadyInd	Add the ReadyInd:peripheralId in the peripherals list.
Error	Error	*	*	—
Operational	Error	Peripherals	FailureInd	—

Local functions

This component has no local functions.

UI Controller

Controls the user interaction by interfacing with the TouchDisplay and the AudioHandler.

Context

Constants

This component has no constants.

State variables

Name	Type	Description
image	byteArray	The PNG image currently displayed or to be displayed.
numSounds	uint	Number of sounds stored in the audio device.
sampleData	byteArray	Sample data of a sound.
soundId	uint	Sound to play or that is playing.
soundMap	uintArray	Maps a touch ID with a sound ID.
touchedAreaId	uint	ID of the touched area.
touchMap	uintMatrix	Mapping the pixels in the image to touch area IDs.

States

State	Description
Initializing Audio	Initializing the audio device.
Initializing Display	Initializing the touch display.
Interacting	Interactive content is shown on the display.
Operational	Ready to interact.
Uninitialised	Waiting for the device to become ready.

Choice-points

This component has no choice-points.

State diagram

State transition table

Source	Target	Interface	Message	Action
●	Uninitialised	—	—	—
Initializing Audio	Initializing Display	Audio	ReadyInd	numSounds = ReadyInd:numSounds. Call the TouchDisplay:initialize().
Initializing Display	Operational	Display	ReadyInd	Send WirelessTransceiver:sendContentDimensions() with the ReadyInd:width and ReadyInd:height as parameters.
Interacting	Operational	Content	SoundInd	Call TouchDisplay:unresponsive() soundId = SoundInd:id sampleData = SoundInd:soundData
Interacting	Operational	Display	TouchedInd	touchedAreaId = touchMap[TouchedInd:x][TouchedInd:y] soundId = soundMap[touchedAreaId] Call AudioHandler:playSound(soundId). Call TouchDisplay:unresponsive(). Call WirelessTransceiver:sendTouchEvent().
Operational	Interacting	Content	ContentInd	image = ContentInd:image Call TouchDisplay:display(image). touchMap = ContentInd:touchMap soundMap = ContentInd:soundMap
Operational	Operational	Content	SoundInd	soundId = SoundInd:id sampleData = SoundInd:soundData Call updateSound().
Uninitialised	Initializing Audio	Device	ReadyInd	Call AudioHandler:initialize().
∗	Uninitialised	Device	ErrorInd	Call TouchDisplay:unresponsive(). Call the TouchDisplay:displayError().

Local functions

Name	Description	Steps
updateSound	Update a sound on the audio peripheral.	Call AudioHandler:updateSound()with soundId and sampleData as parameters.

Handlers

Handlers wrap external resources (timers, network endpoints, hardware peripherals) so the components can interact with them through interfaces and function calls.

Wireless Transceiver

This handler takes care of receiving and transmitting data packets on the radio front-end by interfacing with the radio hardware peripheral. By looking at the message-type of incoming messages, indications are forwarded either to the Connection or Content interface.

Function	Function description	Parameter	Type	Parameter description
connectServer	Connect to the indicated server.	serverId	uint	The ID of the server to connect to.
disable	Disable the radio frontend.	—	—	—
initialise	Initialise the radio frontend.	—	—	—
rejectServer	Block further requests from the indicated server.	serverId	uint	The ID of the server that is to be rejected from now on.
sendAdvertisementPacket	Send a single advertisement packet to become discoverable.	—	—	—
sendContentDimensions	Inform the server of the dimensions of the content the display supports.	displayHeight	uint	The height of the display in pixels.
sendContentDimensions		displayWidth	uint	The width of the display in pixels.
sendTouchEvent	Send the touch event information to the server.	touchAreaId	uint	ID of the area that was touched.

Peripheral Handler

Takes care of detecting, initializing and configuring hardware peripherals, e.g. by accessing low level drivers and serial communication lines.

Function	Function description	Parameter	Type	Parameter description
start	Start initialising and configuring the peripherals.	—	—	—

Touch Display

Handles the interface to the touch display to show images and capture touch events.

Function	Function description	Parameter	Type	Parameter description
display	Display the supplied image.	image	byteArray	The PNG image to be displayed.
displayError	Display the default error message.	—	—	—
initialize	Initialize the touch display.	—	—	—
unresponsive	Disable the touch sensitivity to allow for handling the touch event first.	—	—	—

Audio Handler

Enables the ability to play audio cues to the user upon touch events. It stores audio samples internally, which can be updated via the interface.

Function	Function description	Parameter	Type	Parameter description
initialize	Initialize the audio device.	—	—	—
playSound	Play the specified sound.	soundId	uint	The ID of the sound to play.
updateSound	Update a stored sound.	sample	byteArray	Sample data
updateSound	Update a stored sound.	soundId	uint	Index of the sound.

Type definitions

The types referenced by parameter and state-variable declarations are defined below. Type names in tables elsewhere in this document link back to these entries.

Name	Description	Specification
byteArray	A one dimensional array of bytes	uint8[]
byteMatrix	2-dimensional array of bytes.	byte[][]
uint	Unsigned integer	32-bit numerical value representing an integer value.
uintArray	One dimensional array of uint	uint[]
uintMatrix	2-dimensional array of integers.	uint[][]