Clusters, Grids and Cloud

Clusters, Grids and Clouds -  these are the terms that we are hearing right now at many places. What are these actually ..
First of all let me tell you about a distributed system. Here goes the def

" A distributed system is a collection of loosely coupled systems connected via communication networks "

To say in simple terms, a distributed system is a collection of computer systems that do the same work with each computer having a heterogenous or homogenous processor and memory.

A cluster is a collection of computers forming a distributed computer system connected via fast Local Area Networks.

A grid is a collection of computers which are more loosely coupled and geographically dispersed than clusters. It can be viewed as a large scale form of cluster computing.

A Cloud is a mammoth collection of computers or data centers that provide access to shared resources, applications etc., with out the need for storage or installation space or processor power from your computer. 

Booting

Today let's discuss about booting and what actually happens when a system boots.

During the early days of computers, the boot program is made read only and is put in hardware chips. This boot program is called a bootstrap in computer terminology. The bootstrap initiates all the CPU registers, device controllers and finally loads the operating system into memory. The advantage of this scheme is that, since the bootstrap is read only, no virus can effect it. But the disadvantages outweigh this. First, if we want to add any external hardware,  it's device driver needs to be loaded when the system starts. But, since the bootstrap is read only, we need to change the hardware chips. Also, the operating system must be in a fixed location as specified in the boot strap, if we install the operating system in a different area, then also there is a need to change the chip.

So our computer scientists thought over this and came up with a simple solution. Instead of making ROM's with complete bootstrap program in them, they started making ROM's with tiny bootstrap program's called bootstrap loader. And the main bootstrap program is stored in the disk. Since changing the bootstrap program simply requires a write to the disk, our problem is solved. The bootstrap program on the disk is generally stored in first sector of the hard disk called boot sector.

But... a new problem arrived -- Boot sector virus

Enough of computer terminology, now I will explain in simple terms. BIOS is generally present in non-volatile ROM chips as you know, and this generally initiates the main memory, CPU registers, device controllers and starts the tiny bootstrap loader that is present in it. The BIOS is built at the time of manufacture of a PC and when you start the PC, it is the first software that normally executes, even before the main memory or disk gets initialised. It looks as below

The full bootstrap program as I told earlier, is stored in first sector of harddisk called boot sector. After initialisation of all aspects, the BIOS loads this program to main memory and hands the baton to it for execution. This boot sector in windows computers is called Master Boot Record or simply MBR. MBR contains the full boot code, along with a partition table. This partition table contains the information of partitions along with information whether an Operating system is present in each one or not. If only one OS is present, it is shown or by default it's loaded since there is no need for user to make choice. If more than one OS is present, the user is presented with options to load what ever operating system he wants to be loaded.

Last, about boot sector virus. To prevent boot sector virus, there is an option in BIOS, which makes the bootstrap program on the disk read-only. When ever you think you are not going to install any more OS, check it on and when you want to change your OS, check it off. Happy now...virus problem also solved

Formatting of disk

In the previous post, we have seen that a Hard disk drive or a magnetic disk is a collection of platters covered with magnetic material which are grouped into cylinders. Today let's see how the formatting of disk is done.

Initially when the disk is manufactured in industry, it contains nothing except platters with some magnetic material on them. In the factory itself, low level format or physical format happens. First each platter is divided into sectors and appropriate data structures are written into them to enable the disk controller to read or write data. These data structures contain a header, trailer and a data area where the data is read/written. The header and trailer contain information about how to read/write data, along with a ECC(Error Correcting Code). When ever the data is first written on to the sector, from the bytes present in the blocks of sector, the ECC is calculated and updated. When another read/write happens it's ECC is recalculated and compared with the previous one. If both are different, then the ECC may give two kinds of errors in regard to the corrupted bits

1. Soft error which can be treated by correcting code
2. Hard error which cannot be treated in any way. Normally those bits are ignored and no read/write happens to them. The information about these bits need to be stored in header/trailer to make sure no disk I/O happens to them.

Now the disk is fitted into the computer system and sold to vendor, who tries to install Operating system in it. Here is the place where cylinders come into picture. First he partitions the disk, like one disk for Operating system, the other for user files and so on. Each partition is a group of one or more cylinders. These partitions are nothing but your c: drive, d: drive, e: drive etc., In the next step the Operating system needs to install file system in each of these partitions (NTFS, FAT etc.,). Installing of file systems is called Logical formatting which is similar to physical formatting except here, the data structures related to file system I/O are written into sectors.

If you are a user of Linux, you must have already encountered something like "Swap" space. The swap space is created as a partition but no logical formatting is done to it. It is entirely treated as a sequential array of logical blocks, used when ever needed. Most database systems also tend to use swap spaces since these are free from from file system related overhead.

Also the partitioned disk with a file system is called a Cooked disk and the one with out any file system is called a raw disk.

Hard Disk Drive

Most of us have hard disks in our computers now which are used as a secondary storage devices. As you know that secondary storage devices have non-volatile memory while Primary storage devices like D-RAM have volatile memory.

Ever wondered what lies inside the plastic case of your HDD and how it works. I will show you how..

The above figure is the internal structure of a typical hard disk drive. The hard disk doesn't contain a single disk but a bulk of them which look like CD's and are called Platters. The surface of each platter is logically divided into tracks which are inturn divided into sectors. The both sides of platter are covered with magnetic material. The information is magnetically stored on these materials. There will also be read-write heads used for reading or writing data. These are attached to a disk arm, that moves all the heads as a unit. All the tracks at one arm position form a cylinder.

Transfer time is the time needed for transferring of data between the hard drive and computer. It has two parts

1. The time needed to move the disk arm to the desired cylinder called seek time and
2. The time needed for the desired sector to rotate under the read-write head called rotational latency

Did you ever come across a term called head crash. Head crach occurs when the read-write head makes contact with the disk surface actually. The read-write head just flies on a thin cushion of air above the platter. Although the platters are covered with a thin protective layer, there may be times, when the head damages the magnetic surface. Head crash is irrepairable and the disk must be replaced.

Text editor - Visual basic

Today, let's see the making of our first system software - text editor. We will create it using Visual basic language and Visual Studio IDE. Incase, you don't have Visual Studio, download a free version of Visual Studio or Visual basic express from the below link

http://www.microsoft.com/visualstudio/en-us/products/2010-editions/express

I have made it using Visual Studio 2010, so it may be compatible with 2008 edition, but before that I cannot guarantee. Below is the download link for my project. If you know Visual Basic, it's very easy and intuitive to understand the commands and if you have any doubts, please post them as comments

http://www.filefactory.com/file/cd55136/n/Dileepad.zip

It does all the things that a notepad does, along with good exception handling. The only change is that, it uses " Rich text " files, instead of traditional notepad files. And one more thing, it's name is Dileepad

I have tried to implement spell checker functionality in it, by using "Keyoti Rapid spell Desktop", but it's a paid version and since the version expired, I have to remove the functionality. It's just a single one line in Form_Load, that makes the rich tex box we are using, to gain the spell checker functionality. Try to do it, if you couldn't comment, I will help you. The Dileepad looks as the one shown below

Click to enlarge

If you know Visual basic, skip the remaining post.

If you don't know Visual Basic, don't bother yourself, it's one of the easiest languages to learn and with .net frame work, it has become more subtle. Just go through the code and try to understand it. If you got doubts, clarify them by going through the msdn blog. It has hundreds of articles and forums dedicated to Visual Basic. Or alternatively you can post them over here.


I will give some hints for the beginners to grasp the code. Unzip the file. Open Visual Studio. File->open->Project/Solution. Navigate to the unzipped folder. If the unzipped folder's name is dileepad, go to dileepad->dileepad->dileepad and now you must be seeing a folder with the first subfolder as bin. If not  navigate into one more dileepad subfolder. Finally there will be a Visual Studio Project file named "Dileepad" in the same folder which contains "bin" subfolder. Click on it and the whole project get's opened.


Form1.vb is the heart of the system and it contains the main interface of notepad while form2 is just created to tell about the software for "About" or "Credits". Form1.vb contains usable controls that I dragged and dropped from the tool box on the left hand side. And the things that the control does when a user presses it, is presented in the code behind it. Just double click the control and you can see the code behind it. Very simple and easy to grasp. To run the file, just press "F5".


I have two more system software codes to give you, they are of pure C-language and what those softwares are is a suspense for now.

Operating Systems - Device drivers

You install XP or Vista or windows 7 in any computer, the prime thing that is done, immeditely after installing the operating system is installing of device drivers. Ever wondered what exactly are they ?

" A device driver is a piece of software, that acts as a translator "

You may look at some of my previous posts and there I told you, a compiler is a translator. So, a device driver converts one language code to another language code, hence's it's also a typical compiler.

Consider, we insert a pendrive, the first thing that the operating system does is check for the device driver. If it's not present, then the OS installs it, you can see the pop up on the left bottom corner showing "Installing device driver software" and when it's successfully installed " Your pen drive is ready to use" pops. The reason, why the operating system first checks and does this, is because of the fact that device drivers helps it, to talk with the hardware of the device. Consider the computer passes a high level instruction "copy file - mog.avi". Now our intelligent device driver, converts this into a language that the harware controller that interfaces the pendrive with the computer understands and the task gets done.

Before the invention of device drivers, the task that needs to be done, needed to be partitioned and carried onto different hardware, since there is no communication between them. But now, with a translator, the task can be done smoothly.

Almost every piece of hardware, that is connected to the mother board has a device driver software and they get installed as soon as they are connected. And now a days, most of the device drivers, come bundled with operating system. Windows 7 has almost removed the need for all drivers, except the wifi, and one or two others. And Ubuntu from the beginning had no need for them except the wifi and graphics drivers.

Operating Systems - 3

In case you missed part-1 and 2, below are the links for them

http://systemspro.blogspot.com/2011/06/operating-systems-win-or-loss-of.html

http://systemspro.blogspot.com/2011/06/operating-systems-2.html

Also, the code for different problems related to operating systems, that you come across in your under graduate studies, is made available in the link below

http://operatingsystemscode.blogspot.com/

Today, let's discuss about Kernel

Kernel is a piece of software, that generally comes bundled with operating system and is more closer to the hardware. It provides an abstract layer for the user programs to interact with the hardware and allocate apropos resources to them. In short, it is the heart of any operating system.

Windows operating systems use Windows NT Kernel which was developed and released along with NT based OS in the year 1993, that made windows, a more generic player in the operating systems field. It's success was so huge, that Microsoft never dared to remove "NT" from it's OS names. But in 2000, with the release of Windows 2000, it dropped NT, but gave it a tagline "Built on NT technology". With XP, it dropped that tag also. Presently windows operating systems like Windows 7 and Windows Vista use a hybrid version of the same kernel.

Linux distributions like Ubuntu, Red hat use Linux Kernel, that was first developed by Linus Torvalds in 1991. No one thought that a college kid could invent it. Even the professor who taught him, damned it. And the remaining story, need not be told..

Another Linux kernel called "Hurd" was in the development stages, before 1991, but it  never reached it's completion due to the invention of Linux

Internet bubble or dot com bubble

Today, let's speak of a partly off beat topic.

Before I start, there is a common place, where this dot com bubble happened. Many businesses were born in that area. I think it's some place in california, I am not sure. So for time being, I will use "silicon valley". 

Everyone tells you, that internet bubble got blasted in the early 2000. But no one tells you the reason, I will tell you.

" Netscape "

You may not have heard of this name, but 16 years ago (1995), it captured the whole browser market share - 100 %

I will not tell you the story, after how Microsoft introduced IE, sold it along with their OS and did some other secret things, until Netscape's market share fell to 10 % by 1999 and eventually how the company got closed.

The inventors of netscape, were just a bunch of kids from an American university, who were funded by a person, taking them to silicon valley and eventually got huge profits from it.

Ebay (1997), the same story, kid going college, starts an acutioning site, people throng to it. A venture captalist invests and finally gets profits in bulk. 

Amazon is some what a different story. It was started and run by a completely planned and business minded individual, during the Ebay time.

Amazon and Ebay, when they went for IPO, made huge profits. 

One more common thing - Both were websites. 

This has started an enormous competition among the venture capitalists, who would invest more and become billionaires overnight. And college kids, started coming to silicon valley, envisioning a future, with as simple as a website.

Yahoo's share rose like never before and a lot of websites were born, and the bubble started growing. The capitalists, in liu of the profits, forgot the term "intelligent investing" and started believing every kid, who comes to them, with a website. 

And suddenly the bubble got blasted. A lot of money evaporated, a lot and lot in stock........

This is from a single documentary I remember, from a series of four documentaries I have seen. Accurate information is available in that, and I have shared the Amazon link for you of the documentary for you. They are really excellent.

Don't forget to hit the "+1" below this post

Compilers - 4

This is the last post on compilers. Today let's take a careful look on how GCC compiler works.

GCC Compiler :
Before I start, let us look back into how GCC came into existence. GCC stands for GNU C- compiler. It is a compiler, developed as part of the GNU project.

What is GNU ?

GNU stands for GNU is Not Unix. It is a project started by Richard Stallman, to create a Unix like operating system in 1985. In those days, the source of the Unix is proprietary, so they thought of making a Unix like operating system, with it's source code open. Before the creation of the OS itself, they envisioned how it might be and parallely started creating software for the same. One of the brain child of GNU project, that came into existence during that time is "GCC".

A GCC compiler has the following steps
1. Preprocessing is done - macro expansion
2. The expanded source program, is converted into an assembly language program for the underlying machine architecture
3. The assembly language is later converted into a machine language and stored in the form of an object file
4. Finally, the object file is linked with libraries to form the final executable.

Also, during any stage, you can stop the execution and see how the output of each stage looks like. For the detailed commands, you need to do so, follow this link

Compilers - 3

Ever wondered, how a .net compiler works. I will show you how

.NET compiler :
.net is the framework designed by Microsoft for their windows based operating systems. It gives a great edge for software developers of windows with it's software Visual Studio, the best in class IDE that is available, till date. The IDE is so flexible and intuitive, that even a person with no knowledge of a single programming language, can design softwares in minutes.

Unlike netbeans, the .net framework allows us to use the language of our choice in designing software. Two of the famous languages in .net community are Visual basic and Visual C#. But support for other languages such as Visual F# etc., is also available.

Enough talking, let's see how the program you write, get's compiled and finally runs.

We write a program in the language of our choice in the IDE of Visual Studio. Now in the underlying architecture, it contains the compilers of different languages. Consider we have written the program in Visual C#. The corresponding Visual C# compiler takes the responsibility of compiling the program we have written into a language that is well understood by .net frame work called Microsoft Intermediate Language (MSIL) along with some meta data.

The MSIL and metadata are wrapped into a Portable Executable (PE) and are made available for the later stages.The Just in time (JIT) compiler that is present below it, doesn't compile the whole code into native code, at a time, but compiles it on demand during the execution of code. This gives the justification for it's name Just in time


Tommorrow let's see the working of a GCC compiler

Compilers - 2

Different types of compilers
Depending on the number of stages, compilers are divided into two types

1.One-Pass compiler : This compiler does the compilation process in a single stage. Faster, but not a good one for development.
eg: PASCAL compiler

2.Multipass compiler : This compiler takes more than one stage. Slower, but very good for development
eg: GCC

Depending on their functionality, the following are the different types of compilers

1. Threaded code compiler ( interpreter), used in FORTH. It replaces the given strings in the source with the binary code and gives an error, if it cannot do so

2. Stage compiler, that compiles the assembly language of a theoretical machine, like some Prolog implementations. This Prolog machine, is also known as the Warren Abstract Machine (WAM)

3. Incremental compilers, used in LISP systems, that do a step by step compilation of individual functions

4. Just-in-time compiler, used by small talk and Java systems

5. A retargetable compiler, that can be relatively be modified with ease to generate code for different CPU architectures. All the cross compilers such as GCC, are examples of this type

6. A parallelizing compiler, that converts a serial input program, into a form suitable for efficient execution on parallel computer architecture. Example is the most famous "Open MP compiler"


Tommorrow, let's see the details of two famous compilers, GCC and .net in a step by step manner

Compilers - 1

Today, let's talk about, one more interesting topic called compilers. The def goes like this

"A compiler is a program, that converts a source program in one language into a target program in another language"

The source program may be in any high level language like C, C++ or Java while the target program may be a native code like byte code or machine code that can be understood by the computer.

If you want to design a compiler in the future, you definitely should know about the Bible of Compiler Design. The book is called "Compilers : Principles, Techniques and Tools" by the authors "Aho, Ullman". You can check out the Amazon link on the left side, if you are interested.

For today let's see the difference between interpreter and compiler.

"An interpreter is a program that simply replaces the code present in the source program with the available machine code"

Examples of interpreter include the famous "Basic" language interpreter. In that, when ever you do a mistake, while typing the code, you will immediately get an underline, even before you compile the code, telling, you are wrong and some IDE's (Integrated Development Environment) provide suggestions also.

If you are familiar with the SQL (Structured Query Language), it is similar to the update command. Compiler also does the same thing, replacing the code. So what's the difference ?

The difference is that a compiler is generally capable of translating the whole code at a time, while the interpreter does this line by line. Also, the interpreter takes care of mainly syntax and logical errors, while the compiler is capable of reporting runtime errors also, along with the former. For rapid error checking, interpreter is the best option, since most of the errors can be corrected before running the program, but this alone cannot save us from errors. That is the reason, why languages like "Visual Basic" or "Visual C#", make use of both compiler and interpreter for efficient writing of code and efficient run time error checking.

Tommorrow, let's see different types of compilers and their examples.

Linker

I will try to be as brief as possible about this topic

"A linker is a program that links the object files generated by the compiler and makes them into a single executable file"

Simple and straight forward. But let us discuss, some what more ....

 Consider you are installing Adobe Photoshop or Vice City or NFS most wanted game or any other bigger program. By default, the installer installs a lot of files in the Operating systemdrive:\\programfiles\Adobe\Adobe Photoshop CS3 for Photoshop and in similar folders for others.

Now as per our definition, since a single executable can be made with a linker, why all the remaining object files ( Files ending with .dll .....)

This is due to the fact that the whole program's size will be of some 2-4 GB. If all of it is loaded at a time, the RAM may get overloaded, leaving the system in the busy state. So some intelligent people found a solution to this and implemented as above, naming the new technology as "Dynamic Linking"

Dynamic linking, takes advantage of the fact that the program in execution, doesn't need all the related libraries at a time, but needs them when the user enters a specific area or clicks a button. So it just appears that all the files are loaded when the game starts, but the DLL's (Dynamic Link Libraries) are loaded on demand by the game, so that a lot of RAM is saved.

That's all to say about linker.

I err and I guess most of you do the same, when it comes into using of Programme and Program


Actually Programme is a word present in British English, but absent in American. Also Programme refers to a concert or TV show strictly while Program strictly refers to our Computer science program or software. So be careful and don't go into ambiguity while using these two words

Operating systems - 2

Well, first of all, let me announce you, the availability of free code for the assignments you find in the standard Galvin book in the link below

http://www.operatingsystemscode.blogspot.com/

Incase, if you are having the Operating systems course to study, these are the likely code snippets you will be given home work for.

Today, let's discuss some more things about operating systems.This is the def of operating system

"Operating system is a software, that makes the hardware do useful things"

There are many more definitions of it, but let's stick to this.One more thing most computer science students or IT students including me, when we join college is to fascinate of building an operating system entirely from scratch, knowing each and every detail.

If you try to do the above said thing, you will either become a mad person or get addicted to some other things in life.What I want you to learn is that, building every thing from scratch is not needed today :)

For example, let me give you the example of Linus Torvalds.He did not entirely build the Linux as people think.He used internet to connect with people and gather useful piece of code he needed, and using some of the code, that is already generated, he prepared Linux. I am not telling you to copy the code, but to steal it.In a sense, that you make it more beautiful.In the words of Picasso

"Good artists copy, great artists steal"

Learn to reuse the code and build something on top of it, which makes the software more usable or will be better than that software.And this is what, you cannot do with proprietary software.I recommend you to switch to one of open Source OS and start altering their code or the software you install in them.You willl really gain a lot of insight and knowledge into it.

Loaders - 2

Well, let's go into the boring facts of loader, which you need to learn even if you are unwilling.I'll try to make them interesting though



There are three main types of loaders.Their names and definetions are as follows :

Absolute Loader : As the name itself says, it does nothing other than loading

Relocating Loader : This loader, helps in the relocation of program in main memory (just in case a program needs to be moved up and down in memory)

Generally this task is performed by Assembler, but here relocating loader takes care of that



Direct Linking Loader : This loader, helps in loading the linked subroutines dynamically.It means that if a program is made from a lot of sub routines, each subroutine is treated as a separate program that needs to be loaded as and when linked or needed.

Also remember, there are four main tasks of a loader


Allocation - Allocate space in main memory for program

Linking - Join the linked subroutines to the program

Relocation - Change the mnemonics, that change with their location

Loading -Loading the program to main memory in the allocated space


Let's take a quick look of our three loaders absolute loader, relocating loader and direct linking loader respectively for the above four functions


Allocation - Programmer, Programmer, Programmer
Linking - Programmer, Programmer, Loader
Relocation - Assembler, Loader, Assembler
Loading - Loader, Loader, Loader

Let's take a look into the linkers tommorrow and from the next monday, you will be starting to learn to build system softwares

Loader

Let's start with the definetion of loader.

"A loader is software or program that loads the compiled or assembled program into main memory and initiates the execution"

If you are a computer science student, you must have already come across Von Neumann's architecture, if not, here's the def

"The von Neumann architecture is a design model for a stored-program digital computer that uses a central processing unit (CPU) and a single separate storage structure ("memory") to hold both instructions and data " (source:Wikipedia)

For those of you who couldn't grasp it, it simply states that a program must be stored in primary memory or RAM, that is seperate from secondary memory or harddisk, inorder to be executed.



This thought revolutionarised the computer industry, and to this date, computers are built based on this architecture.Also this is an important interview question often tossed by interviewers, but the candidates are always strike out for this.So keep this thing in your memory.

Although today's computers made more complex architectures for their own purpose, the base of all these is the Von Neumann architecture.

I think you already grasped the need of a loader by now.Since the program originally is stored on hard disk and need to be transferred to RAM as per our architecture, loader helps in doing so.

Is my desk calculator a computer ?
Yes, definitely, but the important contrast between it and your computer is the architecture.Your computer uses Von Neumann architecture, while your calculator uses fixed program architecture (a program can be run from with in hard disk)

If a program can be run directly from harddisk, then you may ask, why this Von Neumann architecture, what is the need for loaders and why are you trying to teach us all these..

Well,  the answer comes from history.The first computer ever that was built is ENIAC in 1949. It used fixed program concept.Fixed program in the sense, it's harddisk is prepared in such a way, that it can run only one type of program.If you want to run a different program, you need to rewire, restructure and change the design.It took atleast three weeks gap to run two different programs in ENIAC.

That is the reason, why your computer can be both calculator and wordprocessor or browser or anything, but a calculator can only be a calculator.

Tommorrow, let's go some what deep into the loaders and learn about different types of loaders and their uses.

Operating Systems - A win or loss of mankind

Operating systems play an important role in our day to day life.Although our primary topic is systems programming, let's discuss the operating systems for every two posts, so that it can be on par with other system softwares.

Let's go back into the history.In 1950's, there were only few computers available and that too for defense or scientific purposes.Many people including big companies, thought that computers were meant for that.But the invent of machines like Altair in the late 1970's revolutionized the computer industry.It showed that a computer can be personal, and could serve the needs of common people.The invent of personal computer and Mac, both in early 1980's completely took the industry by storm and with in five to six years, they have shown a new way for industries.In the olden days, to establish an industry, it would require generations of wealth or should be influential to get bank loans.But, Bill Gates and Steve Jobs have shown."Oh, we don't think so".For the story of Steve Jobs and Bill Gates watch the "Pirates of Silicon valley" movie.Below is the link for it.


And lo, the operating systems were born.And they started growing like anything, giving huge profits to their creators and creating a new kind of jobs called software engineers and finally they became a brand themselves.But it's needless to say, because of these so called WONDERFUL things, many people in colleges started enrolling in computer courses.I am also one of them :)

But you know what, there is also the other side of the coin, not many people mention or dare to talk about.Before computers came, majority of the people used to take sciences, physics, chemistry, mechanics as the preferred courses, and so during the time between 1900 and 1980's a lot of good scientists were born and NASA also got quality people who could send man to moon.Do you know when the last manned mission to moon happen ?

It happened in 1972 (Apollo 17)

Looking at the other side of coin, I always fear what will happen to this world, with the lessening of scientists and the people pursuing research as their goal of life like the kinds of Einstein, Edison, Ford, Wright brothers and lot more.

You may say that many people are taking courses other than software engineering, but my say is that most of them are ending as software engineers.Also I am trying to say, other courses also should be given equal preference

Assembler - 2

In the end, we will be making an assembler for IBM - 360.It took me a week to study about it and make it in pure C language, of you are really interested or taking an under graduate course in the coming semesters, I recommend you to buy this book.Here's the Amazon link for it

Systems Programming

IBM 360 is one of those machines that came in 60's and was a very popular machine at that time, because it  provided interoperability between different machines.Also it is one of the main reasons for the development of assemblers, compilers and so on.

Assembler

This is a very big topic to discuss and hence in this post and the next we discuss about it.This is the def of assembler

"An assembler is a program that converts an assembly level language code (also called as mnemonic code) into machine language code and provides necessary information for the loader to load the programme."

In your day to day life of programming and all, you never come across using this system software anytime.This is because of the compiler which does the same thing, but can convert high level languages to machine languages unlike assembler which can only convert middle level (mnemonic) code to low level (machine) code.

If you are using micro processor and have done some coding, the coding you did is in assembly language.After the invention of compiler, even though assembler became obselete, for device like micro processor which require frequent compiling or assembling, assemblers are still used.Remember compiling is costlier than assembling.


Let's go into the history part of assembler.In the earlier days, people used to write the code in machine language involving 0's and 1's or in hexadecimal format, the only language that a computer can understand till date (seriously).The coding was very tedious and people need to constantly refer the documentation or need to remember a lot of code.

Machine code

The remedy came in early 1960's in the form of assembly language, that contain the parts of code that could be easily remembered in a native language like english.To convert that assembly code into machine code assembler was used.Later on high level languages were invented that were very much close to the native languages (C, C++, Java) and compilers to compile them, that's a whole different story

An example of C-language code with assembly code below (Courtesy : CISE.UFL)

That's all for today, let's talk more about assembler with some code segments as examples.If you have any queries or suggestions for improvising the content, post them in the comments section

Macro Processor

Macro Processor is a program that lets you define the code that is reused many times giving it a specific Macro name and reuse the code by just writing the Macro name only.

Generally it doesn't come as a separate program but as a bundle to either assembler or compiler

Note : Please don't confuse Macro Processor with Micro Processor, remember Micro Processor is a hardware device and it's a completely different area of study

There are three main steps of using a macro

1.Define the macro name
2.Give it's definetion
3.Use the macro name from with in the program anywhere to use it's definetion (this step is called macro call)

And the most important thing is that, macro processing is done before the compilation replacing the code in place of macro calls and deleting macro definetions.If you are on an open source OS, this can easily be tested.For others, this is how it happens.This is my helloworld programme


#include<stdio.h>
#define hell printf("\nHello world\n");printf("I am a good boy\n\n");


int main()
{
hell;
}

Now there is are a lot of options in gcc compiler and if you use gcc helloworld.c -E , it gives you the code that is processed before it is passed to the compiler.It includes a lot of things along with the code.But since our main discussion is about Macro processing, let's see how the macro processed code looks like

# 916 "/usr/include/stdio.h" 3 4

# 2 "helloworld.c" 2

int main()

{

printf("\nHello world\n");printf("I am a good boy\n\n");;

}

You can clearly see that macro processing is already done replacing hell with the corresponding macro definition.This is passed onto the compiler which does the remaining job. This is the compiled output


Hello world
I am a good boy



Note : There are a lot of interesting things gcc command does apart from compiling type gcc --help for more info

Gcc logo

There are a lot more uses for macro and don't confuse functions with macros even though both are used for same purpose and if any one asks you the difference tell them, the things I showed you above, function gets compiled but a macro gets macro processed

One important use of these macros is for booting.During the boot time, the system uses a lot of macros to get itself started since they need not be compiled to use, but just macro processing is enough which is a small task

So, for today, alvida guys, thanks for baring with me, and if you have suggestions pass the comments as you like and hit the +1, button below this post, if you want you can post this one to your friend, since I have enabled posting, see the social buttons for all the things you need.

Text editor

Let's speak of the text editor, the most basic of the System Software, what's the need for it and what's the GAG

The basic aim of the text editor's is that of processing the text, so in your every day life, the notepad you see is a text editor, Gedit is a text editor, Emacs is a text editor, Vim, MS Word, Open office word processsor ... any software or programme that can be used to edit or process the text is called text editor.

Mostly from the olden days to till date, these are used to write programmes by devs, which also became the basic tool for constructing different kinds of documents, that you come across in your day to day life.And mostly these come bundled with compilers and various word processing programmes.Consider the below cases as examples

1. Visual Studio's text editor is the most robust one and user friendly built till date, since it incorporates every other system software ( except OS) in it.The best thing about it is, you need not bother about syntax errors, just like the spell checker in MS Word, it underlines when ever you do a mistake ( This is done by a special system software called interpreter that is normally absent in other editors )
2. Open source has a small yet powerful competitor in this arena, the Vim editor, it has syntax highlighting (This is done with the help of different colours ), but no suggestions are shown as VS.And the best thing about this is it is the most powerful and fastest one, if you know the commands (Yup, faster than VS in this aspect).But it doesn't have any built in compiler
3. Other programmes like Codeblocks, Dev CPP, Free C have the compiler functionality inbuilt into them

If you want to learn about the cool things that you can do with Vi editor (normally available in open Source OS), follow this link

http://flylib.com/books/en/1.533.1.24/1/

Inside that there is a link for Emacs editor also

And about Visual Studio, there is a hell lot of people out there, and basically it's not a text editor, but a software to build microsoft software with ease and it has the coolest IDE on earth. If you want to learn more about it, there are official Microsoft videos available  to get you started and keep going.You can even download them, if you wish

http://msdn.microsoft.com/en-us/vstudio/ff459609

And I presume you already know how to use a notepad :)

Let's talk about the Macro Processor in the next post

System softwares - their actual definetions

Well, let me give the exact definetions of the so called systems softwares

1.Text editor : This is the software that can be used to process the text
eg : Notepad, MS Word, Gedit etc.,

2.Macro Processor : This is the software or just a program that lets the user define macros and use them in the remaining part of code.
eg : The #define you use in C-language

3.Assembler : This is the software, that converts the code, written in Assembly language to machine language.
eg : BASIC-LT, Boreland C

4.Loader : This helps in loading the assembled machine code into main memory for processing
eg : Windows MBR/Boot loader that loads the OS into main memory when you start your system

5.Linker : This is useful for linking of various seperately assembled machine codes and generally done before loading
eg : Linkage Editor of IBM 360

6.Compiler : It converts the high level language code ( C, C++, Java ) into an assembled code or it generally performs assembling part also and provides necessary information for the loader to load the program and linker about the external references with in the program
eg : GCC

In the general sense or in the real world examples, none of this comes seperately, but mostly you will se bundled applications. Take for the example, codeblocks or devcpp or Turbo C

You can write the code in the editor, can define macros, compiling can be done and the program can be loaded in place.

And regarding Operating System, we will discuss it at the end of all the postings, since it is as significant as all of these above programs (softwares) combined. And in the next post, let's discuss of the theoritical aspects of each of these softwares in detail

System Software

I am not gonna say any definition for it, since I personally don't know much about it.Here's the simple def, that came to me by experience

A System software is any software, that acts as a mediator between the kernel (Hardware part) and the user and makes their interaction easier.

Def is ok, but what in the real world does this word relate to

In the real world, it relates to the following important softwares

1. Text editor (Notepad, Gedit, MS Word etc.,)
2. Assembler
3. Macro Processor
4. Loader
5. Linker
6. Compiler
7. Operating System

Most of the words except OS and Text editor, may be trivial

to you.They were trivial to me also, six months ago, but now I am a lot more confident on what each term implies and had even built the first three of the above said softwares.I want you also to do it ...

In the next post, let's discuss of the definetions of thes

e softwares and start dissecting Systems ( I mean your discriminating knowledge about them :))

Just a small exercise for you, till I post the next one, see the below picture, and keep guessing what it implies and fill the comments..

Systems Programming - Foreword for this blog

Well, this is kind of joke, living in the web world talking of systems programming.Yeah, I agree with you, since most of the programs and features that are available today are being exported to the cloud and there is nothing to lose or gain from your system and for a simple PC enthusiast it is of no importance. Yes, this is true.

All services are moving towards cloud.But what's the gag ???

The gag is that it is not for simple PC ENTHUSIASTS any more.How ever, the programs need to run some where, even in the cloud.This means, this topic is not going to be any more with regard to systems but should be renamed as server programming. Well, since our academic courses are not going to be changed so suddenly like the web 2.0 or 3.0 or facebook or twitter, it will take a gradual time of minimum 10 years for it.So till then we can have a cook of systems programming.

I will give small, but enough details to get you started with systems programming and at the end will give you enough details about systems software, so that finally you will create atleast two systems software in the end.