Overview
Most researchers realize the importance of accurate presentation and response timing in cognitive paradigms. Based on research in the field we have found that many sources of timing errors can exist even when using commercial products.  What is needed is a straightforward way for researchers to validate their own paradigms in-situ. To this end, they need a “virtual human” which is easily programmable to make responses as a result of various stimuli it detects. As a result, we have developed a “Black Box Toolkit”.  This low cost kit can be used to detect where timing errors occur and to make corrections where needed.  Without such a kit, it is often hard to control for all sources of timing variance and to carry out successful replications.  This can be the case even when using commercial Experiment Generators as there is variation between different pieces of hardware, there can be errors in scripting or other unknown factors can come in to play.

The Black Box Toolkit provides all the interfaces needed to “virtualize” a human in terms of a typical computer-based study. Your own PC (A below) would run your own unmodified paradigm in-situ and a second MS Windows based machine running the Black Box Toolkit (B below) would respond to stimulus and make the appropriate pre-programmed response.  In real time you are able to record screen events, check synchrony between two stimulus types, e.g. visual and auditory, and simulate responses after a delay you define and with predefined response characteristics. By using the BBTK software, you can analyse everything that happened.  For example, you may find that your audio presentation is early by a consistent 25ms as compared with your visual stimulus.  To correct this simply insert 25ms of silence in the beginning of your audio file.  Alternatively, your response device may add a consistent overhead of 55ms to responses. Therefore, you could correct your statistics to take account of this fact.  In some cases, systematic conditional bias can be found and corrected for, e.g. increased response time where trials contain audio and visual stimuli as compared with trials that only contain visual materials.
 

Click image to enlarge

In the photo of the toolkit above a digital tone generator is shown on the left. In the schematic below the tone generator is used for triggering remote voice keys (O). On top of the toolkit is an opto-detector used for detecting visual stimuli. In the schematic below the opto-detector (M) is attached to the screen of the remote system running the paradigm being tested (C). In front of the toolkit, there are two bare wires that form an Active Switch Closure lead that can be used to trigger a response device such as a mouse on the remote system. In the schematic below the Active Switch Closure lead could be attached to a mouse (I), keyboard (J) or response box (K). On the right hand side of the photo is a digital microphone that is used for detecting auditory stimuli. In the schematic below the digital microphone is shown positioned in front of the remote systems speakers (N). Other toolkit interfaces provide for +5V TTL I/O as shown in the schematic (L). These can allow for interfacing with other equipment such as MRI scanners, eye-trackers etc.

A Worked Example
Let us consider a simple visual choice reaction time paradigm. Where either of two images is displayed for a short period and participants have to respond with a left or right mouse button click.  By default once a response is made the image is no longer displayed and a 2 second (2,000ms) blank screen is shown before the next random stimulus is shown.

To test this paradigm you would modify the two stimulus materials so that they had a white block on them left or right.  Next, you would attach two fibre optic leads to the position on the screen where the stimulus images appear.  Then you would tack two switch closure leads to the left and right hand button of the mouse you are using to accept responses.

Then you would program the BBTK Digital Stimulus Capture And Response (DSCAR) software to look for these images, wait 300ms (about average for a human) and then make a response to say 10 trials.  Below we can see the actual “program” that would test this paradigm.
 

Here we have set-up two possible responses.  Line 5 and Line 6 in the upper spreadsheet. These correspond to left and right mouse button and are easily selected from drop down boxes.  The duration for these simulated responses has been set at 75ms.  Each response can have a variable duration if required.

Next we have defined that if we see any events on line 1 or 2 (the left or right fibre optics) then we will wait exactly 300ms before simulating a response on the left or right mouse buttons which lasts for 75ms.

Note that we have defined 10 trials. The first five make a response on the left mouse button and the last five on the right.

Once we have run the data recorded by the BBTK through the data analyser and compared it with the data recorded by the paradigm itself various issues crucial to timing can be examined:

• Responses recorded by the paradigm should be consistent at around 300ms (as that is what was simulated)
• If the paradigm measures response duration they should be consistent at around 75ms
• The should be no difference between left or right image display and left or right mouse response - that is there should be no conditional bias
• We will know if there is any delay between stimulus images terminating after a response is made
• We will know if there is a consistent 2,000ms gap between each visual stimulus

Thus we have an opportunity to either correct the paradigms script as regards timing, alter the stimulus materials themselves or make post hoc statistical corrections to our data. We may even decide to swap the hardware, e.g. if the mouse originally used was too variable!