What to Consider when Starting a Software Project

Why Projects are not as Successful as they can be?

Choosing the right software development consultants

Should you hire Computer Science Majors as Programmers?

Choosing the Right Database

The Pitfalls of Forward Planning

Choosing the Right Programming Language

Report Driven Design

Problem Solving

What is a computer programming language?

A computer programming language is a language that allows us to instruct a computer to perform a certain kind of task. It is a language that a specialized application (a Compiler or Interpreter) can understand that dictates given a set of standard inputs what we want done or what steps we want followed to achieve our goal. A language that allows us to dictate what goal we want accomplished without defining the steps is called a Declarative Language and a language that dictates the steps to take to achieve our goal is called a Procedural or Imperative Language. Nowadays the term declarative and procedural are a bit fuzzy because most languages have a bit of mix of both in them.

A computer/application programmer is an instructor that takes our human spoken relatively ambiguous laundry list of wishes and translates it into instructions that are absolute enough that a given compiler/interpreter can execute what we want without ambiguity. When I say without ambiguity I mean, given a set of instructions (or goals) in this language, if you were to perform the task yourself, the results will be the same given the same input data and if you know the assumptions made by the compiler/interpreter.

Why do we create programming languages?

There are varying levels of programming languages. In fact all higher level programming languages eventually get converted into a lower level language we call machine code.

Programming language development has strived to do 7 sometimes competing things better:

1. Write applications faster - which translates to same goal in the shortest number of human written steps -> usually meaning more code generators wizards and IDEs that write the code for you or a language with many assumptions built-in.
2. Make applications faster - which translates to fewer number of machine code steps
3. Require less knowledge about how the compile/interpreter works - which translates to as close to spoken/written human language as possible or more code generators and wizards
4. Human readable - meaning you can guess with ease what a program is going to do by reading it.
5. Upward compatible - meaning any application you write with the older version of a language can work in the newer version.
6. Can run everywhere you care - meaning its needs can be found everywhere you would care to use it or that your language is downward compatible (can be made to work in an older or a portable device it wasn't specifically designed for).
7. Immediate gratification - usually meaning interpreted or the server compiles it Just-In-Time (JIT)  - no need for compilation just run with the code

As computer processors have improved and gotten better, we have let item 2 slide a bit. Item 6 has shrunk and expanded in the sense that new languages can't be used in old hardware or OS, but hardware/OS is becoming more varied. So yes new applications we write today are empirically slower than they were a decade ago, but we can write them faster and with less knowledge about how computers work and since our computer processors are faster, the empirical loss in algorithm speed is compensated for by the increase in processor speed. As the web has gotten more pervasive and browsers have gotten more powerful, item 6 for many applications has become every web browser we care about because that is one of the few things that is ubiquitous. That web browser may be all browsers if your audience is big, or maybe just one browser and OS if you can dictate the browser and OS used by your audience.

The other items on the list are weighted differently from language to language. The human readable item is a multi-faceted issue because what is easily human readable to a mathematician is not necessarily human readable to the CEO of XYZ beer company or a molecular biologist. This explains partly why we have such a plethora of languages to choose from. Each language makes certain assumptions about what you want to use it for and which audience is going to use it. A language that tries to be everything to everyone will most likely not satisfy anybody comfortably.

Different Levels of programming languages

The one language days

A long long time ago when computer programming was a new concept, there was only one language for a computer and there was only one computer; everyone that programmed programmed in that language and all applications were written in that language. If we trace back far enough we would probably end up in B.C. Greece or 1800's where the language was mechanical in nature and programming involved flipping gears or punching cards, but we shall not venture that far. We shall instead start our story in the 1950's with the first high level language called FORTRAN (short for Formula Translation) which interestingly as old as it is, is still widely used today in engineering and scientific circles. We don't even dare call it a legacy language as we would ALGOL or COBOL , and yet it is even older than these languages.

Why is FORTRAN not considered a legacy language whereas COBOL, ALGOL, and SIMULA are. Because FORTRAN was designed to serve a particular need and a need that very few languages have been able to fulfill better or better yet cared to fulfill better. It was designed to crunch scientific data. COBOL on the other hand was designed to be a Business Language to write Business Applications, create input screens, be readable by people making business decisions and so forth, and most languages that have come after have targeted the same need. FORTRAN on the other hand had the mindset of being a scientific language for developing formulas with steps for tabulating scientific results. The fact that you could use it for other purposes was a mere accident of design. Does that mean that FORTRAN was a better language than COBOL? No, it more likely has lasted as long because the Business Application space is more competitive than the scientific space. In fact FORTRAN has been superseded by some extent by languages like C, R, SQL and LISP.

The many language days and many framework days

Now we find ourselves in the many language and many framework days. The days when it seems that at least one language is sprouting itself every couple of months and is challenging us to think about whether it serves our need more than what we are currently using. We are even accidentally inventing what I call domain languages. A domain language is a language that makes many assumptions about what you are trying to use it for. For example HTML is a domain language because it assumes you use it to create web pages and web forms and, not to add 2 + 2. SQL is another domain language because it assumes you want to use it to query a database, do summaries of data, and update data, - not to paint screens.

As the title implies Choosing the Right Programming Languages we believe that even for a given application, there is probably no single language that is good enough to satisfy all the needs of an application. For example if you are developing a web application, you may need to know 4 languages - how to query a database using SQL, how to display a page using some sort of mark-up language like HTML, some client validation using javascript, and a glue language such as PHP, VB.NET, C#, Ruby, Perl etc that conducts your orchestra.

The main reason that comes to mind why we can't find a silver bullet language to solve all our problems is because optimizing for one facet deoptimizes for others. In order to choose the best languages for what we are doing, we need to use the divide and conquer approach. Divide our application into separate functionalities and for each functionality weight what features are most important to us and which is the best language to get there. This is where frameworks come into play. Frameworks basically define how your orchestra will conduct business. Frameworks define services that can be accessed and dictate how your glue language will access these services. The services may be supported by other languages that are stringed like puppets with your glue language.

Web or Not-Web?

The first question to ask yourself before choosing a set of programming languages and frameworks is whether your application will be a Web Application, Desktop Application, or Mutant Application. As trivial as that question may be for most to answer, it is not quite as trivial in certain arenas. In fact it may require the toss of a coin.

The question tends to be simple for us because the web is the direction we have chosen for ourselves. In fact often if we think a project is best off as a desktop application, we turn the project down (unless it is an MS Access application which we have great expertise in) simply because our expertise in the Desktop arena has waned a bit.

While there are many languages that work equally well for web and desktop work, the programming classes and frameworks you work with are very different from web to desktop so you might as well consider them different languages. 

The Web option

Why did, we, as an IS consulting company choose to focus our sights on the web? Because we wanted the following

• An environment that required next to zero deployment.
• Ability to change logic quickly and near-real time

Some other added benefits we got:

• More or less Client OS independent
• Can scale easily to different screen sizes and  if you don't use absolute positioning
• Can support wireless devices with minimal effort

Some setbacks

• Browser incompatibilities - We needed to make our interfaces simple to guarantee they would work for our large audience of web browsers and devices.
• It takes a bit longer to develop a web app than an equivalent desktop app

Where it is not appropriate

• When you need heavy transfer of data
• You have no network or internet connectivity or the data you care about is all local to your pc
• Fast response to click wiz bang something happens. AJAX takes care of a lot of this, but in general is harder to code.
• Heavy GUI use. Think Google Earth.
• Heavy interaction needed with office tools such as word processors and spreadsheets - copy and paste.

Now web application development is not for everybody or every application. Web applications even nowadays tend not to be as rich as desktop applications because they are writing to a dumbed down GUI and are designed for low-bandwidth interaction. Also the GUIs for developing web applications are not as streamlined as they are for desktop development. The reason is that for web application development, you want your pages to be as light as possible, and WYSIWIG development suites tend to put in a lot of unnecessary code that makes the wysiwig design easier, but at a loss of weight and ability to adjust to many widths of screens.

The Desktop Option

The Desktop option is most applicapable to the following situations

• No network or internet connectivity
• All data you care about is local to your pc
• Web interfaces annoy you with their limitations of drag-drop, to the point of not being usable
• Your app needs to interact seamlessly and in a very deep way with other desktop Office products

The Mutant Application

An interesting option is the Mutant Application and is one that is gaining traction. A mutant application is one that has a client-side desktop app and a server-side web application. Sort of like a web browser talking to a web application but more controlled. The client-side desktop app tends to be a bit of a dummy terminal designed to communicate in a domain language a little bit more domain specific than HTML. In certain cases it offers some advanced features. These are what Microsoft calls Smart Clients. Although Microsoft Smart Clients framework tends to be very fat at the moment. Another framework is MacroMedia's (now Adobe) FLEX framework. FLEX apps are basically mutant applications built on Flash. Then there are a whole swell of application frameworks built around AJAX that promise to make web-apps more like desktop apps. One popular one that comes to mind is RICO.

Service Oriented Architectures (SOA)

Mutant applications are often referred to as applications built on a Service Oriented Architecture (SOA). An SOA is an architecture by which a server-side app usually speaking SOAP, XML, REST or a very custom application lingo receives messages from a desktop or web client and sends back usually some sort of XML blob that can be consumed by the client. SOA is really an umbrella term for loosely coupled services consumed by clients. It doesn't presuppose any programming language, OS or client or server device.

In short - the Mutant Application promises the best of both worlds - fast/slick response and consumption of various services, but at the cost of in general more complexity under the hood or more stuff to know which is often vendor and task specific.

Pick your Glue Language And Framework

Staff Comfortability

Once you've pinned down the Web vs. Desktop vs. Mutant question, the next thing to decide on is what languages your staff feels most comfortable with. If you have no staff then you probably want to go with a language that is fairly popular that you can find consultants proficient in it.

Lets say your staff is used to a language that is old and not very nimble to change anymore - Like COBOL for example. How do you find the best language for them to learn and embrace?

Basically all languages come with a certain snobbishness or rather (the language designer's preference). Once you figure out what that snobbishness is and figure out if you can accept it or need to move on, then using a particular language becomes easier.

For example PHP and Perl use arrays heavily and value speed at which you can just run with it. Arrays are the holy grail for those languages and they are for the most part not strongly typed. PHP defers from PERL in that PHP tries to be human readable where as PERL tries to be geek/computer readable and brief. LISP likes to do cooky things with functions - passing them around in lists and applying them to other lists; very powerful but very cryptic to an untrained eye and questionable if you need this kind of power in most business applications. C and C++ are a little too in love with pointers, but has great speed and can be made to run anywhere using various compilers. Java is for philosophers who like to talk about whether errors should always be handled and the tenents of object orientation or people who use it because everyone coming out of college knows it and there is a breath of work available in it. C# is for converted philosophers who want to have the choice of handling or not handling errors. Python and Ruby are for people in love with logic and programming at its shortest but who also believe in structure at your command - they are those who used to program in PERL and LISP. VB/VB.NET is for people who just want something that is human readable and gets the job done for most common tasks reasonably well and have surrendered to the evil empire that is Microsoft.

What you want to do is compare your staff's snobbishness against a given language's snobbishness and find the best match, all else being equal.

I would say the hardest thing about learning a programming language is not learning the syntax of the language, but rather the classes, functions, and recipes you use to get the job done. Becoming a master of a language is a little trickier and usually means you have embraced the essence of the language so that you can devise your own recipes to get a particular job done and you know the strengths and weaknesses of your poisons of choice.

Selective Inertia: Think first and then follow the crowd

If you are building an application which is fairly mission critical or has strict deadlines, then its probably best to follow a tried and true glue language for your application unless your application requires bleeding edge technology. The reason for this is

• Many of the bugs/issues you will run into have been worked out
• There probably is a whole ecosystem that has been built around it - e.g. PHP has a strong support network, lots of free tools and a fair amount of commercial and source code to borrow. ASP.NET has a lot of very feature-rich commercial tools (more than PHP from experience, also a lot of free source code (not quite as much as PHP from experience but getting up there)
• A lot of users are using it which usually means a lot of free and commercial support

These are inertial effects that make sense to embrace. The above is not enough. You want to go one step further with Selective Inertia, meaning you want to understand why a crowd of people has chosen a particular path instead of just following the biggest crowd around. When you really talk to people, you find that they are in the crowd because they were following a perceived crowd that became a realized crowd. A perceived crowd is one developed by marketers to make you feel like if you don't follow - you will be left behind. This of course becomes a realized crowd because a lot of people fall for it. I call this Market Hypnosis. Then there are the small crowds within the bigger crowds that have followed a path because they did their homework and mapped their needs to what was available.

What you want to do is listen to the crowd with real reasons for following. For example a person may have tried other options and found - this does not work well in this environment, it doesn't scale, or the others were too difficult to learn. A person might have picked this language because there was a great and inexpensive Integrated Development Environment (IDE) for the language and this is very important when charting unchartered territories. You want to match your applications needs with the voiced needs of a crowd of people. The best way to field this is via newsgroups. Join some of them for languages/frameworks you are thinking about and ask questions. You can also ask questions of a consultant experienced in various languages and frameworks you are thinking about.

Stay away from End of Life products

When you have an end of life product such as ASP, no matter how much the temptation is to stay with it, you should move on for any serious new project. It will only cause you heartache in the end.

Chart of Options

Below is a chart of Glue languages, embedded languages, and Frameworks we've come across and our general impression of them.

Legend
U Usage - D - Desktop , W -Web, Sh- Shell Scripting | L (level of difficulty to learn for beginners -meaning how quick you can get useful stuff working) followed by our rank | M (level of difficulty to Master - how quick you can learn to do difficult projects that are not mainstream) followed by our rank (1 easy, 2 good, 3 medium, 4 difficult) - (If you are a master doing difficult projects can you do difficult projects in it (1 yes easily, 2 yes but a few obstacles, 3 yes but many obstacles note this has a lot to do with the features of the language as well as the code you can easily borrow),

• OS (Operating System) - L - Linux, W Windows, U - Unix, M - Mac, ? means we are not sure
• Syn (Syntax) - C, Cobol, Dbase, Basic, Pascal
• T (Typing) - S - Strong, N-Not typed, L-loosely typed (loosely and not typed languages are often scripted and great for prototyping or getting out applications first but require greater care in programming to minimize on bugs at runtime. L is usually a compromise that tries to get the benefits of N without its nasty side of runtime bugs)
• NC - No Compilation step Required + non-required, - compilation required, * can be run compiled or uncompiled and may depend
• Sp (Speed) (1-4 1 slowest, 4 fast)

a * means it can but not well supported , + yes its supported well

Making computing easier: Programming Languages	Provides a brief history of the origin of computer languages
Computer Languages Timeline	Provides a great family tree-line that shows where computer languages started and what other languages they gave birth to.
Wikipedia Definition of Declarative Languages	Provides some example declaritive languages as well as what is the meaning.
Scripting: Higher Level Programming for the 21st Century	This article was written a while ago, but gives a good summary of the difference between System Languages such as C++ and Scripted and dynamically typed languages. It goes into detail in what environments each is good for.
Building Web Services the REST Way	Defines what Representational State Transfer (REST) is and why it is not a standard but really an umbrella term.
Wikipedia Definition of Service Oriented Architectures (SOA)	Defines what an SOA is and how it fits in with REST RPPC etc.
Rico	Rico is an AJAX framework that can be used inside any web language such as PHP, ASP, Java Server Pages (JSP) or ASP.NET.
Introduction to Ruby o Mac OSX	This provides a nice summary of Where Ruby came from, what its objectives were and also provides a sample program written in Ruby and in 2 other languages
Why Perl?	Some thoughts from a proponent of Perl
LINQ Project	.NET Language Integrated Query is a way of querying all kinds of data such as relational, XML, file system as part of a .NET language so that you get the benefits of Intellisense and syntax checking.
The Hundred Year Language	Paul Graham's take on the evolution of computer programming languages.
Self-assembling Circuits Using DNA May Represent Next Computer Breakthrough	This I believe to be the next revolution in circuit design and will profoundly effect how we program. First we had gears, then vacuum tubes, then electrical silicon circuits, and next we may be using DNA to design circuits.
Programming Languages are like women	Comical caricature of programming languages that depicts what they would be like if they were women.