Concurrent Programming is Scary

by Jonathan

Read Hacker News, Stack Overflow and various programming blogs long enough and you’ll run up against the ever popular opinion that concurrent programming is hard, scary and dangerous. Most articles suggest avoiding it whenever possible. I’ve messed around with pthreads and Java Runnable classes but never really experienced the horror that can arise from concurrent programming. Well, I have now have my own little story that will make me appear wise and sage-like in a programming interview.

As part of my introductory systems class I had to implement parts of a user-level threading package. Most of the work had to do with writing reader/writer blocking locks using spinlocks. There was a small test program to check if the implementation was working and after 20 minutes or so I had things working. “No problem,” I thought, “I’m cruising.”

Condition variables came next, and the code was similar to blocking locks. CVs are associated with a monitor (blocking lock) but have some interesting functions associated with them: wait, notify and notify all. Calling wait on a condition variable will block the running thread, pushing it onto a waiting queue associated with that CV/monitor. Notify blocks the current thread and attempts to run a thread in the CV’s queue. At this point I slowed down and read through lecture notes and my monitor code very carefully. This is the “cerebral” part of the assignment.

My small test program ran as expected after a few tweaks, so I moved on to the last part of the assignment, using monitors and CVs to implement a synchronized stack. Again, the test to make sure everything was working was provided and consisted of a couple thousand push and pop operations in multiple threads. I had to write the push and pop functions that would wait and notify at appropriate times, i.e. it can’t pop when the stack is empty.

Deadlock! Deadlock everywhere. I spent three hours reviewing all my code, even backtracking into monitors and spinlocks trying to track down bugs. A few hundred printfs later I received an email from my professor, an update on the lab. There was an error in his code for push. I removed one line (which I thought looked odd but never looked at again) and my code ran just fine. All tests passed.

What did I get out of this assignment? Concurrent programming is scary. Debugging regular code is hard enough. It’s not a horror story worthy of HN but it gave me some insight. The assignment and fix were simple, and it was done in a lab environment. But if my professor hadn’t informed the class about the bug, how long would I have spent debugging my own code? Probably all night.