64-bit Computing, the Future and You

We’ve known it was coming for a few years now but it only just seems to have sunk in, 64-bit is the way we’re going and there’s nothing we can do to stop it.

The main barrier to 64-bit computing isn’t with any of the incompatibilities or changes to the way things work, though those did play a significant role in preventing uptake.  No, the main thing stopping people from fully accepting the world we’re moving towards is that they have no idea why we need to change in the first place, let alone what those changes actually mean.

Just recently there has been a surge of questions like Can I legally downgrade from Win 7 64 bits to Win 7 32 bits? and How to check whether my hardware is 64-bit capable in Windows? as well as Windows 7 64-bit – More compatible with 32-bit apps than previous Windows? which just shows the general confusion and uncertainty where 64-bit is concerned.

What is 64-bit and why do I need it?

When we talk about 32-bit or 64-bit what we are actually talking about is the width of the registers in the central processor that are responsible for storing data and addresses for a running program.

In order for a program to be able to get a piece of information from memory it first needs to supply an address for the data. It puts the address in a “register” and then it calls a command in the processor that will go to that address in memory and return with the value that it found there. The width of the register determines how many locations in memory are able to be queried in this manner.

A 32-bit address register gives a maximum of 2^32 (4,294,967,296) addresses and it is this number that gives us the hard limit of 4 gigabytes of memory per 32-bit process. I’m making the “per process” distinction because it is an important one in the world of memory managed task-switching operating systems, and is something I’ll go a bit deeper into later on.

A 64-bit address on the other hand gives a maximum of 2^64 (18,446,744,073,709,551,616) or approximately 16 Exabytes of memory. An exabyte is huge. 1024 gigabytes would give us 1 terabyte (the average hard drive these days), 1024 terabytes would give us 1 petabyte, and then 1024 petabytes gives us just 1 exabyte. We are several orders of magnitude away from even thinking of machines capable of making use of that much memory let alone becoming necessary, hopefully it will be long enough that we won’t be changing to 128-bit computers in the next few years.

So how did we move to 64-bit?

This was one of AMDs big hits against Intel.  What AMD did was to gain widespread acceptance of their solution by simply evolving the existing processor and expand all the current registers from being 32-bits wide to 64-bits wide and then make it configurable so that the processor would start up in the old 32-bit compatibility mode and then be able to switch to the full 64-bit mode. This meant a smooth and logical progression of systems to 64-bit as it was simply an operating system upgrade that was needed rather than a full hardware platform change as would have been the case with Intel and HPs joint IA-64 architecture.

The move to 64-bit seemed like a good time to add some other features to the processors and that is what AMD did and brought us AMD64 though there were a few casualties: 16 bit code became unusable in the “long” 64-bit mode and a few related functions were also dropped so 16-bit programs could never run alongside 64-bit programs.  32-bit code was still usable and would be able to run “as-is” which meant that most operating systems would run unmodified, which meant that they could simply manufacture these new processors as standard and let people upgrade their OS as and when they wanted to, the support would already be in place.

Intel copied the AMD64 specification and released EM64T as their 64-bit consumer offering. The two architectures are almost exactly the same but have subtle differences that actual users would never see.  Jointly these two specifications have come to enjoy the name of x86-64, x86 being the original architecture and the 64 denoting the move to 64-bit.

Not many systems needed, or could even make use of, the full 64-bit capabilities when AMD first released their new processors onto the market back in 2003. Most were still using Windows XP and had 512MB or 1GiB of memory, 2GiB was considered the absolute height of luxury. 64-bit computing simply wasn’t needed outside of servers and large database machines hence why the first 64-bit processor was an Opteron processor, part of AMDs server line.

Open vs Closed Source

Linux and many BSD clones were the first to be ported to 64-bit and as all the source was available any changes to support 64-bit were seemingly easy to implement and re-compile.

The first Microsoft operating system that home users could use with 64-bit processors was Windows XP 64-bit and as a home operating system it received much derision. It required updated 64-bit drivers for any devices that needed to directly interface with the 64-bit operating system core (also known as the kernel), this included the graphics card, network card, any motherboard devices such as USB controllers all the way down to printer drivers whose problems were that needed to work using new 64-bit interfaces to the attached devices.

This is where Windows and Linux differed. With Linux all the source code for items like network cards and graphics devices were generally open source and readily available. Granted the 3D acceleration for graphics cards required proprietary drivers but to get a system that worked to show a screen the open source drivers did the job. All the kernel interfaces were well defined and visible to the person developing the drivers so any changes to support 64-bit were immediately visible and so changes to the drivers could be implemented quickly too.

With Windows it was necessary for Microsoft to define changes to their own closed source interfaces, push them through internal quality control, consult with manufacturers and potentially go back to the drawing board. It would have taken several iterations before both Microsoft interfaces and manufacturer drivers were both stable and well implemented, the 32-bit interfaces had been around for years and programmers were knowledgeable and confident in how to use them, new interfaces take time to learn and time to iron out the quirks. Linux would have had similar problems, but being open source it is typically much faster to adapt to change as more people have direct visibility of the changes.

The 64-bit Battle

Windows XP 64 could well be considered a fatality of this learning process, along with the sudden dropping of 16-bit support. Many programs in a corporate environment had been used for many years, almost back to the days of DOS and Windows 3.1, with minimal support or upgrades and the loss of 16-bit support rendered these programs useless. I’ve heard anecdotal evidence of times where games worked on XP-32, but experienced random crashes on XP-64. The main problem is that we have no way of knowing whether it is the new 64-bit platform is at fault or if it is the hardware drivers being unstable due to inexperience with working on the new platform.  They could be using 16-bit features that the processor supported while running in 32-bit mode but were dropped in the 64-bit mode.  We’re left with the feeling that we’re just pounding against walls; the 4GiB memory wall, the obsolete software wall, the compatibility wall and as customers that leaves people frustrated.

Vista on the other hand was a whole other kettle of fish, I never heard of anyone complaining about how useless Vista-64 was, everyone was too busy slating Vista itself as a resource hungry Godzilla-like monster looming on the horizon waiting to destroy the world as we know it. The only thing I could take from that was that machines that were powerful enough to need Vista-64 were presumably A-OK with it. When Vista was released there were still relatively few machines that actually needed 64-bit, most had 2GiB of memory by default along with a 512MiB graphics card and still not that many actually supported fitting more than 4GiB of main memory. The problems started when people with 4GiB of memory and a 768MiB (or even higher) graphics card came onto the scene and we started getting questions like Why does Windows only show about 3.5GB of my 4GB+ of RAM? and Using 3+ GB RAM in Vista/Windows7(32-bit)? and other variations, all of which boiled down to the 32-bit operating system using a linear 4GiB block for main memory and requiring all devices to have their memory accessible within that block.  The reason for this problem was (to me) explained best by Dan Rutter when he devoted an entire “Ask Dan” solely to the question of What’s with the 3Gb memory barrier? His main concerns in 2008 was that 64-bit drivers were not always available, but even then there was a lot of pressure on manufacturers of new equipment to make them available as more and more computers were being manufactured that would be actively maimed by running a 32-bit OS.

The first real honest push towards 64-bit came with Windows 7.  By this time most machines had 3-4GiB of memory, graphics cards of 1GiB weren’t uncommon, and Microsoft effectively threw it out to the masses with the open Betas and Release Candidates which allowed not only device makers to work on it but the people who were thinking about using it were actively encouraged to download it and give feedback to both Microsoft and their device manufacturers.  This helped Microsoft gain a massive head-start, they were already up and running with support for Windows 7 long before its real release.  I for one saw that as a great opportunity to try out 64-bit and see what problems it came up with.  To my surprise the only thing that was a problem was an over-a-decade-old sound card that had “issues.”  I could not say I was surprised by this particular fault, the aforementioned card had issues with me putting more than 2GiB of memory in the computer so 64-bit problems were not unexpected but, not long after I junked that card, the manufacturer released a new wave of 64-bit drivers for a large percentage of their older products, I could have kept using it. People were happy with Windows 7-64, gone were the annoying unattributable random crashes, the operating system and drivers were mature right from the start.  Both sides had made the effort to get it right and, by and large, everyone agreed it was a step in the right direction.

Every review I’ve read for Windows 7 has been largely positive, with no real mention of problems or incompatibilities.  The worst there is is a general disdainful warning stating that drivers for “old” hardware may not be available and an implication that if you are using hardware that old anyway you deserve what you get. It’s this general “Why don’t you just upgrade?” attitude that puts people on edge, they’ve spent enough money on this machine already and don’t see why they should spend more.

I like to think of my computer in the same way as my car, they both have maintenance that needs to happen and there are always new safety or performance features just around the corner.  Change a windscreen wiper here, a hard-drive there and don’t forget an occasional tune-up to keep it running smoothly.  Sometimes I’ll just have to replace it when things go really wrong but that’s the way of things, you should always keep moving forwards rather than being left behind.

PAE – Considered Harmful

So far I’ve been ignoring the fact that  modern 32-bit processors have features such as PAE, mainly as for a home user it is akin to getting a live grenade, pulling the pin and just holding it in your hand. Sooner or later you’re going to be wondering why your arm hurts and why you can’t pick up your coffee mug any more.

PAE allows the processor to change memory management schemes and, while each individual application is still working in 32-bit and subject to 4GiB limitations, the system is able to allow the use of up to 64GiB of memory so that the operating system can give each application its own 4GiB of memory.  Microsoft chose to allow this for their server operating systems but left it out of their consumer systems.  I like to believe they did this for the greater good.  Once we have a system that has 64GiB of RAM but an application can only use 3GiB or less due to needing to have graphics cards and other devices visible in their address space, then people will start asking why we didn’t just ditch this stupid system in the first place and let applications use all the memory available.

Just imagine having a system that worked like this and you bought a future graphics card that has 3GiB of memory on it.  1GiB usable by an application and 3 lost to the graphics card?  Not for me thanks.  Granted you may be able to have 64 of these 1GiB applications, but in the long run is it worth it?