Programmer, Developer, Engineer – What’s the Difference?

Recently, my manager mentioned he was trying to decide out how to properly differentiate between software engineers and programmers. We didn’t have much time to discuss it, but another nearby manager suggested that engineers designed software that could then be implemented by programmers. The implication being that the programmer was just following directions, whereas the engineer was actually doing the “thinking” behind the design. This view is very common among larger, “old-school” companies with a traditional waterfall process history, where product development is divided into discrete steps: plan, design, implement, test, release, maintenance.

After 35+ years of professional programming, I knew this wasn’t an accurate description about how real software development works, but I hadn’t really sat down and thought about it enough to figure out why. It was usually just a question of what title to put on someone’s business card and didn’t have any real-world impact on anyone’s actual work. This recent conversation, however, caught my attention and led me to think about this issue in more detail.

I’m going to jump straight to the end and state up front that I have concluded that there really isn’t any distinction between a programmer, software developer, and software engineer. Now, let me explain my rationale.

Several years ago, I came across an article written by Jack W. Reeves called What Is Software Design? This article was a revelation for me. The basic premise of the article is that software is design. This simple concept resolves so many issues with the way many companies have treated the software development process. There is more information in the article then can be explained here, but the basic idea is that programming, testing, and debugging are all integral parts of the design process. Interestingly, the agile practice of Scrum include aspects of this without really discussing it in the same terms. In its theoretically ideal implementation, Scrum teams consist of several skilled generalists, who are each capable of design, implementation, testing, debugging, etc. These are all part of the same process.

Anyone who’s actually tried to produce a complete written detailed design suitable to be given to an entry-level “programmer” to implement without them having to really understand the design knows this doesn’t work. It has never worked, but companies keep thinking they can do it. To produce a design that can be implemented in a “paint-by-numbers” fashion would require that almost every line of code be described in advance, at which point you have essentially written the software. This is analogous to asking Stephen King to write detailed enough instructions for someone else to write one of his books. As soon as you start abstracting the ideas even a little, the book would become the writer’s book, not a Stephen King book. The same thing happens with software—at any practical level of up-front design, the resulting implementation is actually designed by the individual programmers and the resulting source code is effectively the final detailed design.

It is still important to do high-level architecture and design up front, but the software itself is the ultimate detailed design. I equate software source code to blueprints or schematics. The actual manufacturing process is performed by the compiler and linker, following the detailed design in the source code.

This leads to the obvious conclusion that anyone involved in the writing of software, whether they are called programmers, developers, or engineers, are actually software designers. For those who insist that there be some distinction, it would be reasonable to consider these titles as different points along a continuum of software development. A “programmer” is typically a junior developer, who focuses on designing smaller, self-contained portions of the software. An “engineer” would be at the other extreme, and is responsible for larger component or product-wide design. Additionally, there many be a separate “architect” who focuses on system-level design and the interactions between major components. Each of these individuals is designing parts of the product, but the scope of their design is limited based on their experience and knowledge.

Getting Git

Moving from a traditional source control system like Perforce or Subversion to Git seems like it shouldn’t be that hard. In fact, Git is supposed to be very easy to use, so why is it that when I first started using Git, I kept getting tripped up? I had read Joel Spolsky’s Mercurial tutorial (not exactly like Git, but close enough) and thought I understood it. Joel also made it very clear that there was an unlearning process that had to take place, but I didn’t realize how much I needed to unlearn.

Git tries to use familiar concepts in an effort to make the transition easier, but I think it really does more harm than good. You end up with a situation where familiar terms are used to describe things that are similar only at a very superficial level. It doesn’t take long for the differences to trip you up.

I’m not going to focus on the differences between centralized vs. distributed version control systems (that is covered in other places and is fairly straightforward). I’m going to concentrate on which “old” concepts you need to unlearn, specifically branch, changeset, head, and checkout. Whatever you think these terms mean is very likely wrong in the world of Git. Read on and be enlightened.

COMMITs

The fundamental unit of data in Git is the commit. In most systems, when you commit changes, those changes are packaged into a changeset or something similar. Whenever you commit changes to Git, Git effectively creates a snapshot of your entire workspace in its current state. To save space, these snapshots only contain the delta between a “parent” commit and the new commit. Each commit contains a link to its parent commit. In the case of a merge, the commit contains a link to each of the two parent commits used to create this one. Other than the very first commit, each commit has one or two parents.

So, a Git repository is essentially just a collection of commits connected by backward links to their parent commits. Here’s an example with 12 commits and the backward links from each to its parent:

Repository
Repository

Just to orient yourself, the oldest commits are at the bottom and the newest are at the top (some documentation/software shows this inverted, or horizontally).

Perhaps you’ll be surprised to learn that from Git’s perspective, this diagram does not show any branches. Your first reaction may be “of course it does!” In fact, this figure shows three commit history paths, but in Git parlance, these are not called branches.

Branches

So, what is a Git branch? A branch is simply a named pointer to a commit. Let’s add a few to our example:

Git Branches
Git Branches

We now have two branches, “main” and “feature”. The hardest part to understand is that these branches are just pointers to a single commit. They don’t refer to the entire line of changes. For example, although there are two paths leading to “main”, there is no record of which of those was the main branch. Branches can be created, destroyed, and moved without affecting any of the commits. You can also have several branches pointing to the same commit. These combine to allow for easy changes to the branch names, the ability to “retroactively” create a new branch, and more.

You can also see in this diagram that there is not a branch corresponding to commit 8. Once you merge two branches, you can safely delete the extra branch without affecting the repository (other than the branch “label” disappearing). You can even recreate the branch at the same point in the future if you want. The only thing to be aware of, is that unreferenced commits (those without a branch label or a child commit) will eventually be deleted (Git has a built-in garbage collector). So, if we move the “feature” branch to commit 11, then commit 12 would eventually be garbage collected. Normally Git waits some time before deleting these orphaned commits to allow for situations where the user is simply moving branches around and has only temporarily orphaned a commit.

Let’s take a look at a very common situation. Let’s say you are developing a new feature but forgot to create a new branch for your work and accidentally commit your changes to the main branch. After your commit, the repository looks like this:

Retroactive Branch step 1
Retroactive Branch step 1

where your new changes were committed as commit 13. This can easily be fixed by creating a new branch at commit 13, then moving the “main” branch back to commit 10.

Retroactive Branch step 2
Retroactive Branch step 2

Head

The head is a pointer to your “current” branch. So, the head is a pointer to a branch, and the branch is a pointer to a commit:

HEAD
HEAD

The head is used primarily when you commit changes. When you commit changes, Git follows the head pointer to the branch, then follows that to a commit. It then creates a new commit as a child of that one and moves the branch to the new commit:

Git Commit
Commit

Since the head points to the branch, it effectively moves with it. If you want to change the head explicitly, you can do that by doing a checkout.

Checkout

Performing a checkout on a branch does two things: it moves the head pointer to that branch and updates your local workspace files to match the corresponding commit snapshot. If you wanted to switch to the “feature” branch, you can do a checkout of that branch to move the head pointer there and update your local workspace files to match the associated commit snapshot:

Checkout
Checkout

Summary

The key points are:

  • Each commit creates a snapshot that effectively captures the state of your local workspace at the time of the commit.
  • A branch is simply a pointer to a commit.
  • The head is a pointer to the current branch.
  • Checkout moves the head to a new branch and updates your workspaces to match the corresponding commit snapshot.

It is also important to be aware that some Git tools introduce some complications into this. For example, the process of moving a branch pointer (using a reset command) may cause a checkout at the same time. There are usually command options to control these interactions. Another concept that you’ll have to deal with (which isn’t covered here) is how uncommitted changes to local files are handled during these operations.

The best part of Git is that you can try out commands on your local repository before pushing your changes to the server. Even if you completely mess up your local repository, you can simply throw it out and get a fresh copy from the server.

Fair Taxes

It seems that everyone has a different idea of what is “fair” when it comes to taxes. In fact, most people change their definition of “fair” based on whether a service is provided by the government or a private company. For example, most people would consider it unfair for a company to charge someone more to pave their driveway than another person based on how much they can afford, but as soon as we’re talking about public roads being paved, it’s considered fair.

Rather than try to pick out specific cases and try to fix them, I think we need to take a step back and completely rethink how public services are paid for. Throughout this process, we must consider what is truly “fair” as well as what behaviors the government is trying to encourage or discourage in the population. It is also important to remember that government services are for the benefit of the country (or state, city, etc.) as a whole and not to the benefit to individuals or groups of individuals.

A general rule that applies to almost all taxes and services is that the tax should be tied to the service in some way. Using a gas tax to pay for retirement benefits wouldn’t make any sense—as soon as the price of gas drops, the retirement account goes into a deficit. These types of situation usually arise when there is money to be “found” in some other area that can be redistributed to meet some goal. There are some definite places where this breaks down, in which case we need to look deeper into the objectives of government.

Let’s start looking at specifics. As you read this, don’t worry about the poor, disabled, children, etc. I will cover them at the end.

Basic Government Services

Fire Protection: Fire protection is basically a form of insurance (in fact, it was provided by insurance companies in the past). There are basically two things being protected by the local fire department, real estate and personal property. It’s fairly easy to see that the fire department should be paid for by taxes on those two things: real estate taxes, and sales taxes on “durable goods” (the fire department isn’t there to save your consumables like toothpaste, napkins, milk, etc.). In both cases, the value of the fire protection services is almost directly proportional to the value of the item being taxed.

Police Protection: The police protect the same things as fire protection, plus they protect individuals, plus keep general order in society. Like the fire protection, some portion of the police should be paid for through property and sales tax (a much smaller portion though – the police generally aren’t protecting a house from total destruction). The protection of individuals is a bit trickier to value. While it could be argued that some people (the President, company CEOs, scientists, etc.) are more valuable to society than others, most people don’t think that way. So, if we assume every person is equally valuable (and also assume the everyone benefits equally from the “law and order” function of police), then ideally each person should pay the same amount of tax for police protection. Since there really isn’t anything that can be taxed that would produce the same amount of revenue for each person, we’re left with a straight $x per year line-item on our tax returns.

Water and Sewer: These are both fairly easily paid for as a billed service. Metering water usage also works as a reasonably accurate proxy for sewer use.

Parks, Recreation, Public Spaces: The value of these can be divided into two primary groups. First, is the community as a whole—the benefits of these projects lifts the entire community by raising property values, reducing crime, attracting business, etc. The second group comprises the individuals who actually use the parks, ball fields, etc. The second group can be accommodated through admission charges at major parks, equipment rental and lighting charges, permits for fairs and festivals, and the like. The first group is much harder to value. The first consideration is that these benefits are regional (a neighborhood, city, county, state or the entire country depending on the specific space). The question then becomes how to tax each individual in the affected area an amount corresponding to their individual benefit. A logical choice is to base this on income—the general idea being that some portion of a person’s success can be credited to the “environment” created by these government expenses. Similarly, a property tax may be appropriate as property values are correspondingly affected.

Roads and Bridges: Like parks, there are two primary beneficiaries of roads and bridges. The obvious one is the people who actually use the roads. The other is the public as a whole. The benefit to the public as a whole can generally be split evenly among the tax payers. Like parks, this benefit can be covered by income taxes. Like parks, individuals can credit some portion of their financial success to the availability of roads and bridges. For the users, it is even simpler. The users of the road have the biggest benefit and also do the most damage that needs repair/maintenance. This can be covered by a combination of fuel taxes (a rough proxy for amount of use) and vehicle taxes (different types of vehicles do more damage to the roadways). Ideally, these two taxes would multiply together, but that isn’t really practical currently.

Education

Education is an area that is difficult to value and pay for. This is because students are not taxpayers and the value to each student isn’t really tied to any easily taxable expense. Speaking only of how we pay for it (not how it is spent), income tax seems to capture some of the idea. Like some of the above taxes, it can easily be argued that high-income individuals can credit a portion of their financial success to the education they received as a child. This ends up being a process of each generation “paying back” the government for their education and in-turn funding the education of the next generation.

Without going too much into the spending side of the equation, it stands to reason that every child in the country should have an equal education, regardless of their community, financial status, etc. This suggests that education should be funded as a national level and distributed evenly per child. There may be additional, unequal, funding at the school level to cover physical requirements like busing, heating and cooling, security, etc. but these regional difference should be funded separately from the actual per-student money.

National Defense

National defense is very similar to police protection in that there is the individual protection that everyone receives for their person, real estate, and personal property as well as a general national benefit (security, way of life, etc.). Obviously the amount of money is greater, but the benefits are also weighted more heavily toward the protection of the nation as a whole, with the individual benefit being almost a side-effect of that. As with many of the other “benefits everyone” services, it can be difficult to determine who benefits more (and therefore, who should pay more). There is obviously a large amount that benefits everyone equally (life, liberty, and the pursuit of happiness), but many people wouldn’t be able to pay for an equal share in national defense. It is logical then, to credit a large portion of an individual’s financial success to the national defense (more directly to our system of government that our national defense protects). As such, it make sense that an income tax would account for the varying levels of benefit. It also makes sense to include a sales tax, since part of what is being protected is our economy and our freedom to spent our hard-earned money any way we see fit.

More research would have to be done, but if a sizeable portion of our national defense was to provide a constant supply of oil, it would make sense to have a tax on oil-based products (gas, fuel oil, etc.) that contributes to the nation defense budget. This same concept would apply to any other commodities or services that could be taxed in a similar way.

Healthcare

It is important for this discussion to realize healthcare isn’t really a government provided services (except for VA hospitals), they just “manage” the national forms of it. This blog post is much too short to cover healthcare in any depth, so the discussion will be somewhat limited.

First, it is very important to distinguish between healthcare and health insurance. Healthcare is all about the day-to-day health and medical needs that are shared by everyone. Health insurance, on the other hand, provides for rare or unexpected medical needs that only apply to a portion of the population.

The basic premise of insurance is that everyone pays into a pool, from which “emergency” funds are available for those that need it. Insurance works by virtue of the fact that most people use only a small portion of what they put in. This allows for the few people that have major expenses to use a disproportionately large share of the pool. This is generally acceptable to everyone because they know those funds will be available to them if needed.

Insurance breaks down very quickly if it starts covering things that most or all of the contributors use. For example, if everyone gets a physical exam every year, there is no benefit to using insurance. Each participant ends up pulling an equal share for these expenses, but at the same time, the insurance company has to take some of that money to cover the expense of managing the system. It makes more sense for individuals to simply pay for their own “normal” medical expenses directly without involving a middleman.

A more controversial problem is that of pre-existing conditions. Insurance works by because the probability of any one person needing some service is very low (if there is a 1:1,000,000 chance that each person will have a $1,000,000 expense, the effective cost to each user is just $1). A pre-existing condition has, by definition, a 100% chance of occurring. Taking the earlier example, if just a few people with $1,000,000 conditions join, the cost to the other users starts going up rapidly. They are no longer paying to cover the risk of an event, they are simply paying someone else’s bills (pure redistribution of wealth).

Healthcare plans are designed to cover both the day-to-day and pre-existing condition expenses. Since there are no probabilities in either case, there is no real money to be saved. The day-to-day expense part simply works like a forced savings account, and the pre-existing condition part is a forced wealth redistribution. In reality, a large portion of the population can’t even afford the day-to-day expenses, resulting in the redistribution of wealth to apply to those cases, too.

In the case of insurance, participants contribute voluntarily knowing that they could have expenses that far exceed their individual contributions. In the case of healthcare, a large portion of the individuals can never get more out than they put in, and some portions will always get more out than they put in. In general, this would be an unacceptable imbalance, so what happens is that healthcare and insurance are combined into a single plan. Since these plans are still unbalanced, there usually has to be some government mandate to force people to use such plans (a standard insurance plan plus paying for your own day-to-day expenses should always be a better deal for someone without pre-existing conditions or who wouldn’t benefit from the redistribution of wealth aspect of combined plans).

Another big problem with both systems is that the individual making the “buying” decision is no longer the direct payer, the plan is. This results in a breakdown of the fair market system and results in prices increasing (there are no market forces to keep prices down). The “standard” solution to this problem is to add regulations to try to keep prices in check. As is usually the case, this sets up an ever-increasing battle between the private healthcare industry and the government.

So, what is the solution to this problem? Unfortunately, there isn’t a good one. The fundamental issue is that a large portion of the population can’t afford the healthcare they need, so the government must appropriate money from those who have it to cover the difference. The issue then becomes deciding what system is the most “fair.”

The amount of healthcare required by an individual isn’t really connected to their income, wealth, consumption, or any other financial transactions. The challenge then is to find things that are close. First, it can easily be argued that a portion an individual’s financial success can be attributed to their health, so a portion should come from income taxes. Next, you can look for “nearby” expenses, like cosmetic surgery and other elective procedures. These could be taxed (fairly heavily) since they directly benefit from the healthcare system as a whole, and are entirely voluntary.

A more controversial idea includes additional taxes for families who have more than two (or some other number based on research) children. This has two positive effects: it reduces the load on the system by reducing the number of children born to families that can’t afford their healthcare, and secondarily, it would provide extra revenue from those family who chose to have more children. This could also be a progressive tax, where each additional child has a higher tax.

Social Security

Social Security is one of a few government programs in that it doesn’t really have any service behind it. It is simply a mechanism for transferring money from currently employed workers to those who have reached retirement age.

There are several ways for individuals to have income during their retirement. The easiest is to simply save money. Without any sort of investments, you would have to save a large percentage of your income, and would have to work many more years than your retirement years. By investing the same money and earning a reasonable rate of return, the amount of money being invested goes way down and will allow for a much longer retirement period.

There are, of course, several problems with plans like this. The first is that most people don’t have the willpower to voluntarily set aside a large portion of their income every paycheck. It always seems like the money is needed for bills and the like. Second, any investment plan has risks—over the long term, the risks are fairly minimal (even significant losses will be erased in time), but in the short term (for example, the last few years before retirement, or during retirement itself), there may not be enough time for losses to correct themselves.

Social security works on an entirely different mechanism—working people pay a portion of their income to the government, which then gives that money to the retired. The current workers’ benefits will then be paid by future workers. The theory is that this provides a guaranteed flow of money from one generation to the previous and will continue to work virtually forever, and as the economy grows, the retirement benefits effectively grow along with it.

What it can’t guarantee is the amount of the retirement benefits. Since there is no investment or other “growth” mechanism built in, the system can only pay out what it takes in each year. Let’s take a simple example: assume everyone makes $100,000/yr., each worker contributes 5% of their income ($5,000/yr.), and there are four times as many workers are retired people. In this scenario, each retiree receives $20,000/yr. (minus any overhead). This isn’t great, but it is something, and most of the retirees’ contributions were made when incomes were much lower (i.e., they didn’t contribute $5,000/yr.).

The big problem comes when the number of retirees increases. This can happen due to population fluctuations (the so-called Baby Boomers) or to extended lifespans. If we take the previous example, but assume there are only two works for each retiree, then those retirees can only receive $10,000/yr.).

There are really only two ways to fix this: 1) increase the contributions by workers, or 2) decrease the number of retirees. Option 1 has obvious limits, based on what workers can actually afford to contribute. Option 2 can only really be done by increasing the retirement age. This, make sense for other reasons, too. Retirement is an opportunity to enjoy the last few years of life without the burden of having to work.

Let’s look at an example. Suppose someone works from age 20 to 65 (45 years), then retires and dies at age 85 (20 years of retirement). First, it is obvious that expecting 65 years of income for 45 years of work is “challenging”. Now assume, due to life expectancy increases, that the worker lives to 90 years old. Increasing life expectancy by 6% adds 25% on to the retirement years, without increasing the contribution. Realistically, individuals should work an additional 6% (2.5 years) to make up the difference.

As to the issue of fairness, each worker’s contribution should be exactly proportional to the contributor’s income throughout their working life. This makes it an easy income tax. Note that there should be no upper or lower limits to ensure that each worker’s retirement will be the same proportion of their working income.

Welfare (including food stamps, etc.)

Welfare is another program that doesn’t really have any service behind it. The reasons for providing welfare fall into two major groups, compassion and societal productivity. In general, the government shouldn’t be making decisions based on morals or compassion. That’s not to say those are a factor (ignoring it can result in unrest), but those reasons are better served by civic or religious organizations who choose to help those in need on a voluntary basis.

The primary reason for the government to provide welfare services is to help maintain a stable society that can grow and prosper. By providing a foundation for each individual’s economic life, they are better positioned to become productive members of society.

The challenge, of course, is to determine how best to fund welfare in a fair and efficient way that promotes productivity, stability and growth. This is another area where it can easily be argued that individuals with higher incomes can attribute a portion of their success to society as a whole. As such, most of the costs of welfare programs must come from income taxes.

Another possibility is to actually employ low-income individuals in government positions. This addresses several problems at once, by immediately making these individuals productive, and a portion of their benefit/income could come directly from the taxes collected to pay for that service. In many cases, these individuals would still be earning more than their direct value (either because there are more workers than jobs, or because the workers are not trained or able to work as efficiently in the position they are given), so additional subsidies must still be provided by taxing the income of workers (again, their income represents the proportional benefit they have received by having a stable society).

Summary

Many of the major issues in our tax debates relate to progressive vs. regressive taxes, tiered taxation, ability to pay, etc. Throughout this post, I have attempted to eliminate those issues by tying each government program to a tax paid by the individuals who benefit from the program, proportional to the benefit they received—this is inherently “fair.” Having said that, it might turn out that lower income individuals simple can’t afford the level of public service they are using. There are really only two solutions to this problem:

First, we might determine that the government is simply doing too much. If someone making $20,000/yr. is using $10,000/yr. of government services, we have a fundamental flaw in our whole government and economic system. This is simply unsustainable—either the government services are too expensive or this individual is taking advantage of more government services than they actually need. In general, the government should only be providing essential services that benefit the country/population as a whole.

Second, we could decide to make the system unfair by taking more money from individuals that have it in order to pay for the services used by those who can’t afford it. Some would argue that this is fair because a high-income individual obviously get more benefit from government than the low-income individual. That is true, however, it has been taken into account in each of the areas of taxation I have discussed and should be considered for other benefits that I haven’t included. If a high-income person truly benefits (or benefitted) from some service more than a low-income individual, then that service should have an income tax component that results in taxes proportional to the income.

Hello world!

After several false starts, I have finally set up a “permanent” home for a blog. Contrary to my earlier instincts as a software engineer, I am going mainstream and using WordPress. My previous attempts to develop a semi-custom (or an least customizable) site resulted in my spending more time working on the site than on the content. I’m going to try to focus on content this time.

One thing I will be doing that is against the recommendation of most “how to do blogging right” experts, is that I will be blogging about several unrelated topics. Hopefully, a few of my readers will find something of interest in more that one category. If, however, you want to stick with one, I will be putting my posts into several different categories. This list will change over time, but for now, I have set up the following categories:

Software Development
My full-time job and primary hobby is computer programming. Over the years, I have worked in many languages on many platforms writing everything from OS kernels, to desktop applications, to web sites. My current work is primarily in the area of desktop and web applications using Microsoft technologies. This includes C#, .NET, WPF, ASP.NET MVC, JavaScript, jQuery, HTML5, CSS 3, etc.
Electronics
I started dabbling in electronics when I was still a child. My first “real” job was as an Automatic Flight Control Systems Specialist (autopilot technician) in the U.S. Air Force. Starting in the 1990s, hobby electronics became somewhat more difficult, as the world was moving toward surface-mounted components, DSP, custom ICs, etc. More recently, there has been a real resurgence in the hobby electronics world (a.k.a. the Maker movement). Extremely low-cost microcontrollers like those used in the Arduino and Raspberry Pi boards (and many others) has opened up a world of possibilities for fun and educational projects that would have been too expensive or complicated just a few years ago.
Pro Audio
A couple of years ago, I picked up a part-time gig as a sound engineer for a local band. That band is no longer around, but I still have a decent PA system and occasionally provide sound for small solo acts. I have a pretty solid understanding of audio, acoustics, electronics, etc., but am still new to real-world sound engineering. I have tinkered with home recording in the past, so there may be an occasional post about that, too.
Politics
I try to avoid politics in everyday conversation, but I do have many thoughts on the subject I would like to share. I’ve started an outline for a book to be called “Reinventing Society” where I plan to take a close look at the role of government in society. I will post various parts of that effort here to get some feedback from readers. Hopefully you will find it enlightening.

Over time, I plan on adding some separate pages to cover some of my previous and current software projects, as well as any Windows 10 and Windows 10 Mobile apps that I release (they’re in the works…more details later).