Trial setter and text-based UI

I've written the 'trial setter' that I described in my previous post. To recap, I had previously built tools that allowed primitive user interaction with the arduino: the user could echo commands to the arduino by typing them at the command prompt. For instance, typing echo "FORCE R" > TO_DEV would tell the arduino to deliver only stimuli that mean "choose the right port". We use this in the lab to force the subject to alternate between choosing left and choosing right, in an early stage of training. But it can be tedious to type in all of these commands manually, so why not automate it?

That's what the trial setter does. It reads the history of events in 'ardulines' and sends commands automatically to the arduino. For instance, it will force right trials until 5 rewards have been obtained, then force left trials until 5 rewards have been obtained, and so forth.

Here's what that code looks like:


It's a bit funny-looking, because it uses those persistent global variables. That's because the idea is that something calls this update() function over and over again, as often as possible, and on each call it needs to remember its current state (for instance, which side was most recently forced). 

What's going to call that update() function over and over? And what if the user wants to change the number of rewarded trials necessary to trigger a switch? We commonly start with long strings of rewarding the same side, and then decrease the length of this string as the subject improves. To address these questions, I wrote a simple text-based UI using the curses module.

I had never used curses before, but it's a really easy way to make simple text-based UIs. It's also pleasingly anachronistic, at least to me, and reminds me of the way I used to write BASIC programs in the 90s. Later on I'll upgrade this to something graphical, but that's a whole new can of worms.

Anyway, the UI provides a menu of options which can be accessed by pressing a single key. Some of them then require additional input. For instance, the screen shot shows how a user can change the number of rewards necessary to trigger a switch, by first pressing s and then typing in 25 at the prompt.

I also included shortcuts for other common actions, like manually rewarding a certain side or echoing arbitrary text to the arduino. Now the user doesn't actually have to type in echo commands in the terminal, which actually ends up saving a lot of keystrokes and aggravation in the long run.

This is all a layer of abstraction above the language I had previously implemented for the user to communicate with the arduino. Basically, it takes in user-friendly actions ("press R to reward the right port") and converts them into text commands that are easier for the arduino to interpret.

Here's the main code for doing this:

And a few helper functions:

Curses simplifies the processing of receiving keyboard input and writing text to the screen. The tricky part, as always, is the timing. Currently it waits for the user to press a key, but only up to a second. Then, since nothing is happening, it calls the update() function described above to run the trial setter. Then repeat this process forever. Once the user actually presses a key, it enters a blocking mode as necessary, to receive additional input for instance.

The next steps are to build in some more interesting logic in the trial setter. As the behavioral task becomes more complicated, we'll want to do more than just alternate sides. I'll probably also put some time into packaging up all of the different components into something cleaner and better documented.

Data flow chart

This is a flow chart of the way data flows through the entire system, from the subject to the experimenter. Currently, everything is working except for the automatic trial setter. That's the next step: a script that intelligently reads the session history and sets the parameters for the upcoming trials. At the moment this is either done manually by the experimenter (for example, setting rig parameters) or within the Arduino itself (for example, choosing the stimulus to present on each trial).

A better user interface for viewing hits and errors in real-time

I've written a simple Python script for scanning the 'ardulines' file and producing a continuously updated display of the subject's hits and errors.

Here's a screenshot of the display:

The trial number is on the x-axis. Only the last 80 or so trials are shown, to avoid crowding the points.

The top 8 rows are different varieties of "go left" trials. The bottom 8 rows are different varieties of "go right". Red is error, green is correct. (Rarely, the subject refuses to answer at all, producing a black "spoiled" trial). The row labels on the left show the individual percent correct for each variety of stimulus.

One thing that I noticed in the behavior was a very strong "stay"-bias. That is, if a subject went left on the previous trial, he's likely to go left on the next trial ... regardless of what the correct answer was on either trial! This is obviously a non-optimal solution to the problem. With time, subjects typically learn to suppress this poor strategy.

In the mean time, it's useful to be able to diagnose which strategy the subjects are using. To do this, I fit an ANOVA to the subject choice on each trial (LEFT or RIGHT), as a function of the correct response, the previous response, and a bias term reflecting a constant preference to go one direction or the other.

These values are displayed (admittedly, a bit tersely) in the title of the figure above. The factors are called "rewside" (correct side); "prevchoice" (previous choice); and "Intercept" (side bias). For each factor, I've shown the fraction of explainable variance (EV) in the subject's response that is encoded by that factor, as well as the coefficient and p-value for each factor.

This subject is doing okay (60% correct); fortunately, the rewarded side is the largest determinant of his choice. Stay bias accounts for a small fraction of the variability, and there is no side bias.

Serial port latency

Today I noticed some unexpected delays between state transitions. Although these delays were relatively small (tens of milliseconds), they were troubling for two reasons:

• We aren't doing anything particularly intensive on the arduino yet, so why would it be delayed? Since the arduino runs at 16MHz, we have 16000 clock cycles per ms. This should be plenty for everything we're doing now.
• Down the road, we'll want to ensure we have maximal temporal precision. <1ms would be nice. I suspect this will be a recurring issue, so I wanted to get some practice dealing with it.

Here's the specific case in which the delays occurred. To deliver rewards to the subject, we use two states:

• START_REWARD : a digital output is activated to open a solenoid valve. Also, we note the current time and add 40 ms to it. Call this the "timer". Go to REWARD_TIMER.
• REWARD_TIMER : Check the time. If it's greater than or equal to the timer, close the valve and go on to the next state. Otherwise, keep looping through this state.

Each loop through REWARD_TIMER should be pretty quick, since all we have to do is check the time and compare it to the timer. <1ms precision seems very doable here.

But here's the results I was getting. These are the lines taking from "ardulines". The states are numbered instead of named in this output. START_REWARD is state 5 and REWARD_TIMER is state 7.

...

TRIAL OUTCOME HIT

3521 STATE CHANGE 1 3

3523 EVENT REWARD_L

DEBUG reward timer 3563

DEBUG reward timer 3563

DEBUG reward timer 3563

3523 STATE CHANGE 3 5

3644 STATE CHANGE 5 7

3668 STATE CHANGE 7 8

...

Let me walk you through it.

1. A correct trial ("hit") occurs
2. Time 3521ms: the state changes into START_REWARD
3. Time 3523ms: we reach START_REWARD, and this event is announced
4. I added the debug announcement while investigating this issue. It prints out the value of the timer: 40ms plus the beginning of START_REWARD, or 3563ms, as expected. I repeated the debug announcement 3 times for test purposes (see below).
5. Time 3644ms: We transition from START_REWARD (state 5) to REWARD_TIMER (state 7).

Why did that happen at 3644ms instead of 3563ms? That's like 80ms late! That's an eternity even for an 16MHz CPU. It's also screwing up the time that our valve is open and therefore the reward amounts.

Moreover, I noticed that repeating the DEBUG announcement 3 times increased the latency to the state change. This hinted that the problem was in the Serial.println function.

But the documentation for Serial.print clearly states that Serial.print is non-blocking! That means that the Arduino should have continued processing state changes, even if the serial writes were slow.

Well, as it turns out, the serial writes are stored in a buffer which is only 64 bytes long (an amount I consider to be ridiculously small). Once this buffer is full, the Serial.print function becomes blocking: it does nothing until the buffer has been emptied. I strongly feel this should have been documented!

So what to do? As a short-term solution, I increased the baud rate to the maximum allowable: 115200 baud (characters per second). At the default setting of 9600 baud, transmitting 100 characters takes 10.4 ms, plus probably some start/stop overhead. At the higher setting, it should take only 0.9ms. Hopefully it can empty the buffer before the next announcement is made.  

In fact this works quite well:

TRIAL OUTCOME HIT

3584 STATE CHANGE 1 3

3586 EVENT REWARD_L

DEBUG reward timer 3626

DEBUG reward timer 3626

DEBUG reward timer 3626

3586 STATE CHANGE 3 5

3627 STATE CHANGE 5 7

The state change is now supposed to occur at 3626 ms and it occurs at 3627 ms. Not bad.

Another workaround is to increase the size of the serial buffer, but this looks relatively complex.

One conclusion here is that my serial messages should be as short and infrequent as possible, to avoid overflowing the buffer. Okay, I can make them shorter by abbreviating the messages (SC instead of "STATE CHANGE", for instance). But at some point we may have quite a lot of messages to transmit (every single one of the subject's touches, for instance). We may run into a fundamental limitation of the arduino, or really its serial port.

More generally, this is one of the reasons why real-time operating systems exist. If the solenoid is supposed to be open for 40ms, I want a guarantee that it will be open for exactly that long! RTOSs are designed for this purpose, but the arduino is not. Even worse, the addition of debugging information can change the latencies! 

Nonetheless, with careful programming I think we should be able to satisfy a reasonable tolerance (<2ms) most of the time (>99%), and hopefully detect any failures when they occur.

A simple behavioral protocol

Let's put together the tools we have so far and build a complete behavioral protocol. This is for an early phase of behavioral shaping, in which all a subject has to do is activate a touch sensor, after which he will immediately receive a reward. We also want the ability to give rewards at arbitrary times, even when the subject has done nothing.

Here's what will be running on the computer:

1) The chat interface. This receives input from the Arduino over the serial port and immediately writes it to a file on disk. Let's call this file 'ardulines'. Also, it checks a pipe, called 'TO_DEV' for any input from the user/experimenter, and sends it to the Arduino.

2) Another terminal which allows the user/experimenter to write to the pipe TO_DEV. This is the same pipe that is checked by the chat interface; anything written to it is sent to the Arduino. This is how the user/experimenter can signal the delivery of a reward at an arbitrary time.

3) A graphical display script which reads all of the text in ardulines, parses it, and displays the progress of the subject graphically.

I've intentionally separated the chat interface from everything else. That's because we want this to keep running even if something else, like the graphical display, crashes. We also want it to keep ardulines constantly updated with the latest info from the Arduino. Thus, in the worst case when everything crashes, we'll have all of the data from the session.

Later it might be nice to combine scripts 2 and 3, so that the user can both view the progress of the subject and send new signals to the Arduino within the same environment. Let's keep it simple for now though.

Here's what will be running on the Arduino:

The sketch will implement the chat receiver from last time, to catch text coming from the computer (such as user commands). It will also contain a simple state machine that goes through the following states.

• State 0: Trial start. Send the time to the computer. Go to state 1.
• State 1: Wait for input. If subject touches the sensor, or if the text 'REWARD' is received from the computer, go to State 1. Otherwise stay in State 0.
• State 2: Deliver reward. Open the reward valve. Start a reward timer. Go to state 3.
• State 3: Wait until reward timer reaches desired value. Close the valve. Go to State 0.

The nice thing about state machines is that it's very clear what the inputs and outputs are supposed to be in each state, as well as when to transition. That makes debugging easier.

The waiting state 3 is an example of how to avoid delay commands in the Arduino. Instead, we should aim to call loop() as often as possible, even if we do nothing on most of the calls. That way, our other code (such as receiving chats) will still run; if we had used delay, then nothing would happen until the delay was over.

The code

All of the code is available in the LickTrain directory on the ArduFSM project on github. I'll also include snippets here to explain them.

The chat.py module incorporates the chatting functions that I discussed in previous postings, as well as the logic relating to ardulines and TO_DEV pipe.

On the Arduino side, there are a couple of modules -- chat.cpp and mpr121.cpp -- that handle the chat receiving and the touch sensor, respectively. Thanks to Jim Lindblom and SparkFun for providing the MPR121 code!

Here is the loop() function in the main sketch that implements the state machine:

Here is a script, main1.py, which I use to start and run the chatter:

In operation!

I used this to run an experiment. Here's a screen shot:

Here's what's going on:

• Upper right terminal: this is an IPython instance running main1.py, shown above. In addition to writing out ardulines and polling TO_DEV, it is also echoing everything to the terminal as a simple text-based readout of what's going on.
• Lower right terminal: here's where I can write the string "REWARD" to TO_DEV whenever I want to send a reward manually, as opposed to the automatic rewards that the state machine delivers after touches.
• Lower left terminal: another ipython instance where I'm running a simple graphical display script, shown on the upper left. This script parses ardulines and can display graphs based on the events it reads. In this case there is not much to display, so I'm simply creating a histogram over time of rewards. I'd like to have t

Next steps

There's a few user-interface niceties to take care of. It would be nice to have the graphical display script run continuously, so I don't have to update the graph manually. 

I'd like to standardize the text output from the arduino a bit more. Essentially we are creating a communication protocol. Ideally we'd be able to send asynchronous commands (like REWARD) to the Arduino, as well as set parameters (like the inter-trial interval) on the fly so that we don't need to hard code them in the sketch. Conversely, the Arduino should report on everything it is doing (like state changes, touches detected) so that they can be written to ardulines in a standardized format.

Probably most important is creating a more complex state machine, one that implements a standard two-alternative choice psychological paradigm, so that we can study more interesting behaviors.

User interaction with chat interface

Previously I showed how to communicate between Python and the Arduino. Next we need a way for the user to interact with the Arduino, via the Python interface. The user might be a person, typing commands into a terminal, or it might be another script. Ultimately we will want to control the entire behavioral shaping process from another script, which for example will observe the subject's performance and adjust future trials accordingly.

For reading: Let's write out everything the arduino says to a text file. This way we'll have an exact record of what happened. Other programs can parse this file for the results so far.

For writing: This was a little more difficult. I wanted a general interface that could be used by a human or another script to communicate with the Python chatting script. I decided to use a named pipe, which is similar to a file. Anything that is written to this file-like object will be picked up by the chatting script and communicated to the Arduino, line by line.

Here's the old code as well as the new code:

The syntax for reading from the pipe is a little clunky. I wanted it to be non-blocking, because otherwise the program halts if there is nothing to read. But the non-blocking read will raise an error whenever no data is found. So we catch that exception and ignore it.

To create the pipe named TO_DEV, we just need to run:
    os.mkfifo('TO_DEV')

Now you can write to the pipe, either from another terminal like so:
    echo "MESSAGE" > TO_DEV
This automatically appends a newline.

Or from another Python script:
    pipeout = os.open('TO_DEV', os.O_WRONLY)
    os.write(pipeout, 'MESSAGE\n')
Here you have to add your own newline.


Next time I'll package this up into a complete script and demonstrate it.

First step -- a chat interface

The first thing we need is a way to communicate between python and the arduino over the serial port.

Python side

The `pyserial` library makes things easy here.

The `timeout` keyword is a key element here. That defines how long `readlines` will wait for input before returning. If you set it to 0, it will return whatever it finds immediately. The problem with that approach is that we're going to be calling this function over and over again in an infinite loop. I suspect a smaller number of calls with a small delay will end up being more efficient than a much larger number of calls with a near-zero delay. That's something to test layer.

The `write` function actually writes immediately and returns ("non-blocking").

The Arduino side

The arduino needs something to say. Let's have it report the time every second. Also, let's have it acknowledge any chat it receives, simply by echoing back whatever it hears.

The actual `receive_chat` function resides in a library.

Things are more complicated here than in Python because we have only low-level functions available. Whenever there are new characters available, we add them to a global variable `receive_buffer`. It has to be global because it needs to persist over many calls to this function.

Once the buffer gets to a newline, it stops taking characters and returns the line it received. It also echoes it on the serial port so that Python knows its message was received.

What if multiple lines are waiting to be received? This will only return the first one. This is easier than coding up a linked list of multiple commands. It also means that other things have a chance to run on the rest of the update cycle.



Now the arduino can communicate with Python. But how does Python interact with the user? The user might be a real person, or it might be another Python script, for instance one controlling the state machine. The next post will cover the methods by which the user can read what the arduino is saying, and type in responses.