(Editors note. I wrote a series of rants on requirements in a blog a friend and I were working on. It was called “Doug and Doug doing Embedded Design”. We have since taken it down mostly due to a total lack of interest from us and zero traffic. We did it to support a side project building a car mp3 system.

I am putting the rants here, because they are part of my development manifesto. I also need some easy posts as I am really busy for the next few weeks. I have edited the posts and tried to test links. If you see any issues, let me know.)

Building the product requirements

If we are treating this project like a product development effort, where do we begin? The answer in most companies is with the Marketing Requirements Document. Every engineer reading that last sentence just let out a groan of agony.
Don’t get me wrong, as an engineer I think marketing is hugely important. I have worked closely with some very good marketing people. If the marketing people you work with do not know all about this, you have bigger problems than just requirements.
But wait, Joel says we need a document. It is one of the 12 items in the Joel Test.
I can hear your argument being shouted back, “wait you fools! The 37Signals guys say don’t do it”
The truth, as usual is not black and white. Personally I lean towards Joel Spolsky’s way of thinking. You need some way to communicate what you are planning to build to everyone involved.
In the 37Signals case, there are few people involved. They do mock ups and design the UI of their applications before coding begins. That could be called the requirements, but they would object to the word.

In my experience the requirements process is always a failure. I have read and researched the topic. It seems to me that it should be very a useful and good thing to know what I am building before I start. As an engineer, I crave a detailed and complete requirements document. I am always been left hungry.
Why does Marketing get to decide anyhow? We are the engineers. We know what to build! We built the widgets that paid for the building they sit in. For anyone curious about the history, there was an article in the Harvard Business review titled “Marketing is Everything” by Regis McKenna from 1991. A book followed. If you read the article it describes the coming utopia of market driven companies and happy empowered consumers.
Alas, the practice is much harder than the theory.

Requirements are really hard

A recurring them on this blog and I have found in life is the fractal nature of reality. Requirements sound simple, write a document that says what to build. Requirements are an art and science in their own right. As we zoom in, the details are complex at any level.
There is a requirements convention and an IEEE society, plus many books.
Two of the best I have read are Exploring Requirements: Quality Before Design by Donald C. Gause, Gerald M. Weinberg  and Customer-centered products : creating successful products through smart requirements management / Ivy F. Hooks & Kristin A. Farry.
Anything by Gerald Weinberg  is usually worthwhile.

Why the requirements process fails

Here are some of the ways I have seen the requirements process fail. Doug and I will try to avoid these pitfalls in our simple requirements.

Marketing is not able

In the beginning, the founders get together around a great idea. There are few people around the lunch table. They throw out ideas and one is really good. It gets hashed out and everyone understands. They found a company and build the idea into a business.
Now after a while, it is time to build a new product. This will be a really new product, not an add on or update to the original. Of course, the founders sold out or left last summer to spend their dump truck of money they earned off the first idea.
The marketing people are now tasked with writing the requirements. Of course, they have never done anything like this before and no one on staff is a requirements expert. No one has read the books, so the time needed is underestimated. Read that last sentence again, it is the killer point.
Marketing is out trying to setup a time to talk to customers, get an NDA in place before they do, and not piss off the sales people. Because this takes much longer than expected, the engineers who were sitting in on the kickoff meeting get bored. Bored engineers are dangerous, idle hands and all that.

The Engineers know what to build

As engineers, we are industrious by nature. We go and gather our tools and begin to build something. Depending on how long the marketing group takes, we may be pretty far along.
When the document is delivered, deliberated at the highest levels in the organization, and passed down to the engineers the conversation goes something like this:
Engineer: :”We can’t do that”
Marketeer: “Why not?”
Engineer: “We never planned on that feature in the design.”
Marketeer: “Why are you planning and designing without the requriements?”
Engineer: “You took to long, it is always up to us to make up the schedule time, etc.?”
Marketeer: “What did you build?”
Engineer: “The same thing as last time, but we put a clock in it.”
Marketeer: “Arrrrggggg!!!!” Slam (Conference room door slamming and the sound of fading screams in the hall).

Sales is in the drivers seat, not marketing

This is the classic. The sales team does not want anyone grilling the customer about what comes next. If the customers know a newer, bigger and better product is around the corner, they will wait and not clear the old product off the shelf.
So, a smart sales persons says, “I talk to customers all the time, I know what they need, just ask me and we will get these requirements done quick!”. This is the sales person as proxy for the customer syndrome. As an engineer I have to say it is one of the worst syndromes a company can have.
Marketing talks to sales. Sales wants feature X. Feature X is big, ugly and difficult. A fearless engineer, lets call him “Doug” asks, “Why do you need feature X”. It goes back and back through layers and up and down the hierarchy. Finally the answer returns and it is always the same, “Customer A said they would buy $1 bazillion worth if we had feature X.” OK, now Doug asks “How much will they pay for feature X?” The answer, “Pay? no, they won’t pay extra for it.”
This time it is the engineer running through the halls with his underwear on his head screaming.

The customer says ‘whatever’

Once again the company goes out looking for feedback from the customers. When the customer finds out this is not even a real product yet, could get booted at any time, and will not be in their hands for over a year, they just don’t care.
Very few customers have time to tell you what you should be building. Usually they are busy running their own business. Yes, the product may be revolutionary and really save them money and make life easy. The fact that you still have to build it and they must wait is a complete turn off.
Even worse, if the customer realizes they have a need that your current product does not meet, maybe a competitor’s does. Or why buy now? Why not wait for version 2.0?
Most users don’t know what is possible. So it is very difficult for them to have an opinion about the next generation of your product. Most of the feedback is either completely impossible, costs way to much, or goes like this, “Just build the same thing as last time, but put a clock in it.”

Exactly how many customers responded to the survey?

Be very careful when you hear statements like “All the customers need this feature”, or “everyone we spoke to has to have it or they will not buy”.
The flip side of customer apathy is the over interested customer. If most people can not be bothered, there will be one who is more than willing to specify a product that meets their exact needs. Does it meet the needs of a larger market? Well, that is our problem, not theirs.
In more than one company on more than one occasion I have found that “all customers” really meant the one who we met. You can not do statistics with only one data point. Two is not much better. Unless you are selling to the government, there had better be more than a few customer contacts.

Requirements are just hard to get right

Most small and even large companies do not have a requirements expert. I have never actually encountered someone who has read the books I recommended above. Few companies will go to the expense of hiring a consultant, at least not at the beginning.
Most of us muddle through using one of the three methods listed above. Some poor soul is given the task of writing the actual document. It is unfortunate that the result is usually a document. A database or tool could really help.
The document is written in good faith as best as can be managed. It gets sent off to the engineers and immediately rejected. Meetings are scheduled and time is wasted going back and forth repeatedly. In the end, we build the same thing as last time, but put a clock in it. Sigh.
What went wrong? The engineers complain that the document was:

  1. not specific enough
  2. told us how to implement features, not what they were
  3. gave no priorities
  4. was too detailed
  5. had untestable requirements

In the back and forth some important things get dropped, the wrong ones get emphasized and it winds up being design by committee. A generic product is described that looks just like everything else in the market.
If no resolution can be found in the meetings, the project is thrown into development with orders to “just do it”. Maybe a consultant gets hired to sort it all out. See the section about bored engineers to see how this works out.
Rinse and repeat.

Parts two and three coming soon.


How to write Ashton Kutcher into “Two and a Half Men”

What this has to do with multi-core software I just don’t know. This came to me while brushing my teeth this morning. They were talking about it on the radio, so it must have seeped into my subconscious. Now I have to write it out or my brain will be stuck. Some friends know I rewrite sitcoms for fun, why I dunno. If you are not familiar with the changes in the show, they are explained here.

Obviously the writers of the CBS show Two and a Half Men have a problem. Lucky for them there are random people like me on the internet to help. I have come up with a plot that will work, and could be mined for comedy gold. The great fear of all fans is that Ashton will be a direct replacement for Charlie Sheen’s character. I have to agree. The problem is how to get him on the show, get Charlie (the actor and character out) and not jump the shark.

First let’s get Charlie off the show. This is actually simple. Do not do it soap opera style and kill him off, only to lamely resurrect him later. Instead use the now classic sitcom gimmickry, and make Charlie the character we only hear from and never actually see. The character Rose (played to perfection by Melanie Lanskey) is the key. Charlie realizes she is the only woman he has ever really had a long term relationship with, and they go off together to Europe. Now we, and Alan (Jon Cryer) sort of know that Rose probably has Charlie drugged and tied to a bed someplace, but Alan gets to move upstairs in the Malibu beach house,  drive Charlie’s Jaguar, plus keep Berta and all expenses are paid. The setup episode involves Rose announcing all this, Alan going to save his brother, then picking up a beautiful woman in the Jag. He decides Charlie is probably happy with Rose.

Now Charlie can send in a weekly postcard from various spots around the globe. If Charlie Sheen decides he needs a paycheck signed by CBS, he just shows up. No need to revive the dead!

With just Alan in the house, things would get dull fast. CBS WRITERS DO NOT MOVE JAKE INTO THE BEACH HOUSE. Having Jake and Alan live together does not make for good comedy. The whole cast has to get involved for the most fun. But, Alan has to have a roommate, or things get dull. No, not Ashton either.

Jake has to stay in Judith’s house. We should put him into culinary school, so he is racking up bills for Alan, not leaving the house, having kitchen mishaps, and meeting girls from school. There is lots of material right there with his friend (Graham Patrick Martin) in tow it is all fun. It gives a reason for Alan and Herb to go places with the kids, interact with Judith, and sets up plot after plot.

I have always thought Ryan Styles as Judith’s (Marin Hinkle) second husband was, one hilarious, and two not used enough. He held up well on The Drew Carey show, so lets move him in with Alan. The plot is simple, Judith throws him out, he knows Alan is the happy Jaguar driving bachelor in the beach house, so he shows up on the door step. The whole tension over the paternity of Judith’s daughter is good for at least three episodes, plus a nice running joke. It may be fun to have Herb meet gorgeous women, while Alan still can not win.

Now, we never really see that daughter. This must be handled carefully. Yes, the cute kid is a sure sign of shark jumping, so casting is important. The kid should be cute, but has to be believable as Jake’s (Angus T. Jones) little sister. She should be cute, but not too cute. She can share his huge appetite, but be really sharp. A bit more Judith, less dense. Think a young Mayim Bialik, and it will play better than a super cute kid.

What about Grandma? Holland Taylor as Evelyn Harper is absolutely perfect. She shows up, worries about Charlie, but does not do anything about finding him. She eventually gets a clue about Judith’s daughter. Jake is her favorite, then everything flips and Jake can not figure out why. That is one episode I am giving away for free. She shows up, gives Alan a hard time, has a strange, uncomfortable relationship with Herb. Maybe have them wake up in the same bed after a wild night. All good stuff.

Now, where do we put Ashton into all this. (You thought I forgot him.) It is obvious. Ashton can not play a Charlie replacement. He can play himself, the sort of doofy, good looking, younger lover. Yes, he should be Judith’s new love interest. He plays the Ashton character that will feel familiar, but keep some twists so no one gets bored.

Jake and his little sister hate Ashton’s character, of course, as do Herb and Alan. Why wouldn’t they? While Judith is smitten, Ashton makes Charlie look like a choir boy. He chases anything in a skirt. The kids catch on, eventually after some close calls. Then they can plot to get him caught by Judith. Here is the twist, he acts dumb, but is really pretty smart. He is not going to let anything stop the good thing he has going. He is living with Judith for free, totally financed by Alan and Herb. Ashton can use the Malibu house for his hook ups, trying to weasel the use of the Jaguar, and to get the kids. He can find out the daughter is really Alan’s, and tries blackmail, but has to be careful. If Herb finds out, he can get out of alimony to Judith and the free ride ends.

It may even be fun to make Ashton’s character a teacher at Jake’s community college culinary school. That gives him some power to manipulate Jake, but he still has to be careful. Everyone is living a bit of a lie, Herb is paying to support Judith, and Ashton, plus his daughter (if she really is his). Anything to upset the balance and stop the funds will unite the enemies.

Add a bunch of beauties stopping by periodically looking for Charlie, and the fun flows free.

The key is to not dump all this in one season premier. There is a full season just to get through the setup, then a season to let plots play out. We really don’t need a cliff hanger ending until the end of Ashton season three.

So, what does this have to do with writing software for embedded systems. Let’s say if one part of the system starts acting crazy, you may have to rearrange the circuit to make it work. How’s that?

If I get some comments, I will reveal my super secret soap opera twist, and the cliff hanger for Ashton season three.

Me and startup companies

It is no surprise to the readers that I am interested in startups and entrepreneurship. The only people who read this blog know me or are related.

I left a big company ‘X’ to join a smaller company. They called themselves a startup, but not really. The current gig is well funded, if not completely around the corner of profitability. At over 150 people, it does not feel much like a startup.

I have met with friends over lunches and breakfast for years to discuss ideas, new companies in the news, and the entire philosophy of startups. Fun stuff to talk about. We all have a dream (I have watched “Tangled” about 15 times with the kids).

My friend Frank said you have to be a special sort of crazy to found a company. You have to be smart enough to come up with a good idea that can be implemented, yet at the same time dumb enough to think you can actually do it all.

Another friend and I applied to the TechStars program in 2009. David Cohen was kind enough to send us a few emails. We did not have enough traction to get in on just our idea. Still, it was a good idea. I was excited to hear about Chris Piekarski in the TechStar NYC class. I can say I knew him when he was green and just out of school. With a day job and kids at home, TechStars is not the best fit for me.

I have not written about any of it here, because this is for work. I write about technical stuff, mostly to have some proof that I can type words. That is important in a tight job market. Remember, always keep an eye open for the next big thing. You never know when they will move your cheese. So this is the first post under the heading of entrepreneur.

What changed? There is another summer program called the Founder Institute. I made a short video and sent in my application. They asked me to take their test. It was a standard personality test and a pattern matching game. They let me in. When the universe gives you an opportunity, you have to take it, right? What does this mean to the day job, my free time, and my family? At this point we don’t know.

As of now, I am not sure how much blogging I can do about the course work and other activities. The course does not start until May. Right now I am working on the idea, trying to test and refine things. If I get more than two comments I will post my application video. It is embedded software and sort of multicore related.

If you are an embedded developer and willing to help, let me know in the comments. I need to talk to engineers and pick their brains.

Asynchronous Communication

I just rediscovered this post from Oreilly about asynchronous communications. Take a moment to read, I will wait.

It brought to mind another recent article about how multicore programmers must think differently.  The author is arguing that “different” thinking is needed, not that multicore is harder than regular programming.

What relates these two things? Asynchronous communication is multicore.

People attracted to programming are by nature and by training linear thinkers. Step on leads to step two and three, so on to the end. Sequential programs are drummed into our heads through all our college years.

Unfortunately, I am not a linear thinker. I am a leaper. Step one leads to B which jumps to crocodiles, and that means the answer is 7, done. Being a leaper in a linear world is a double curse. It is hard to communicate with the linear thinkers, because either I have skipped three steps and already arrived at my answer, or I do not follow their steps, so can not see the big picture without more detail. Plus, as a programmer, taking to the non-technical is always a bit iffy.

Understanding that the world may not happen step by step is the first step to understanding multicore. Process A may finish before B, sometimes, but not always. The messages may get swapped in transit. We can put things back in order, or find a way that order does not matter.

It is an asynchronous world, and linear thinkers better get used to it.

The myth of the small processor

It is great to be an embedded developer. We are drowning in riches. The job market is good despite the economy (knock wood), and the world we live in is rich with new parts and fun things to do with them.

My recent work is a departure from the multicore and bigger processors I worked with in my past. I have to go way back over a decade since I last worked on an 8051 or 6805. (If you don’t know what those numbers mean, don’t worry about it. ) Recently I have been working with smaller Atmel AVR microcontrollers.

Once again the great myth of small processors has run over me like a truck. Many coders believe the myth. An because they believe it, it continues, sort of like an evil Moores Law.

A good design won’t fit on a small processor. It is wrong, a myth, and it needs to be busted.

Small processors have less memory, less code space and more limitations. That is not an excuse. Actually, it is an excuse, for poor coding and bad design.

So, what gets broken and how do you fix it?

First, fix the state. All programs have states. The person writing the code may think they don’t have state, then the state gets hidden. Much better to put it out into the open where it can be seen. If there is a flag that gets set, that is state. Someplace in the code has a do this before you do that, but don’t do something before the first two, that is state. In a small system these things become globals and flags. That is wrong. Don’t add a flag, build a state machine.

There can be only one. It is good for states too. Flags are considered harmful. Bit wise flags in a global are considered evil! A set of bits in a global variable do not represent states. Every possible combination of flags in the set of bits represents the states. Are you ready to test every one of them?

I recently looked at some code that used flags. All through the code were statements like (if this flag do x). For a tiny microcontroller this causes code bloat and uses up the available memory fast. Much better to build the simple switch state machine with one variable. Don’t set a flag, add a state. Too many states? Not really.

Say the code requires the security message before the 25 other messages. OK, so we add the SECURITY_MSG_RX flag that gets set when it is received and validated. Now every other message handler has to check if(SECURITY_MSG_RX == (flags & SECURITY_MSG_RX)) (yes I do write my bit wise tests like that). No, you say, put the check in the receive handler, not every message handler. Except the security message has to get past the checking. One exception, not a problem. No add two more flags, rinse and repeat. You quickly have a mess.

The better solution is to make a state variable. Call it recvState for this paragraph. Now, set the state to RECV_STATE_WAIT_FOR_SECURITY. That one state checks for the security message and dumps anything else. Don’t move to the next state until secure. All states after can assume the security is valid, no need to keep rechecking.

A nice way to handle the state changes in add a setNewState(_stateType) call. You did make your states a typedef, right? Grep for all calls to the module private (static) call to setNewState() and every single state transition is visible. Add a last state variable and a log message that reads “Changing state from STATE_ONE to STATE_TWO” and life is good.

Another piece of code I could not understand had a buf[BUFF_LEN * 2]. Then all the code checked on BUFF_LEN. Why? If I had to guess, one of the checks was wrong, or misplaced, so sometimes the buffer over flowed. This is a bad fix for bad code that hides problems. Please make the code correct, and then there is no reason to waste space.

One last strategy that helps save more space but I never ever see. Write programs that handle tables. This is not my idea. It comes from Code Complete by Steve McConnell.

Every possible place, build a table. Write code to go through the table, then handle only the parts that are different. This is the object oriented concept of programming only the difference. Once the table handling and common case code is working and tested, adding more lines to the table is easy. For an embedded system declare the tables to be const, static const if possible. The const key word should (compilers vary so take care) keep the table in the program space, not in RAM. Don’t put variables in the table. The resulting code will be much smaller, easier to test and expandable.

This works great for networking. Tables of messages are easy to implement and save code space over a switch on the message ID. Extra points to anyone who comments with the way to find the table size as a constant at build time, in C.

If there are worries about traversing the list, put the list in order and use a simple binary search. For 10 items it is not worth the effort, but every little bit helps.

Here are just a couple of tips that can make life with a small system happy and worth while. Please comment if you have others.

100 Different cores – The minimal risk machine and multiple cores

Frank made a comment to my last post. In my usual totally tangential way, it reminded me of the Minimal RISC (Reduced Instruction Set Computer) machine.

When I was a boy, taking Computer Architecture from Dr. Chang many years ago, the RISC vs. CISC was a real debate. The paper above is for the minimal RISC machine.
The idea is the computer has one instruction. That instruction is move memory A to memory B, done. The computer can do anything with that one instruction. Adding, multiply, everything is in the memory map. To add 2+3 the computer moves 2 into the adder slot A, then moves 3 into the adder slot B, then gets the result from adder slot C. Easy.
The idea of 1000 of these super simple cores is fun to play with. next to the cores would be banks of adders, multipliers, dividers, or whatever is needed.
This makes some sense from a resource point of view. Computers tend to add, subtract and compare more than divide. Things that take a long time in an algorithm like multiplication, could be run in parallel. Start multiplier A, then B, then C, get result A, then B, then do an add and fetch C. Cool.
Just something to noodle on.