Educational Tool
The first application for such an operating system is as an educational tool. Without a collection of reverse engineering tools, it is impossible to examine MS-DOS in any great detail. Anyone who has ever looked at maintenance of any software system will tell you that trying to get into the original programmer's head to figure out their reasons for a particular design, without proper documentation, is a monumental task -- especially at the disassembled assembly code level. My complements to all the folks who are pouring out reams of "undocumented documentation." Reverse engineering MS-DOS may also break any existing licensing agreements between you and Microsoft, so check your license before applying a disassembler to it.
My guess is that MS-DOS source code is almost entirely in assembly language and the kernel itself guaranteed to be assembly language. Even if you did see the source, it would be more difficult to follow than if coded in a high level language. My goal of minimizing assembly language yields two benefits: the code is portable and easier to understand. Although a high level language in itself does not necessarily guarantee portability, it is definitely easier to port C source code than assembly source code. Also, conditional code and looping constructs are definitely easier to understand in C than assembly language.
As an educational tool, you can replace any section of the tested operating system. This allows you to experiment with different operating system algorithms. You may want to change the LRU (Least Recently Used) buffer algorithm for a different one and compare performance. You may also want to learn about the different algorithms in order to apply similar technique to a different application. For example, loadable device drivers may be useful in a stand-alone application where the host uses different devices in different configurations. Another example is the buffered keyboard input code may provide enough insight for an interactive display system.
Finally, you can test the code on a host MS-DOS or UNIX machine using a source level debugger because the majority DOS-C code is C. For example, performing DOS/NT code tests on a 680X0 based UNIX machine proved invaluable. Many of the "big-endian" versus "little-endian" issues surfaced quickly during the unit testing of individual components of the operating system.
Embedded Systems
Possibly one of the most useful applications is as a n embedded operating system. Many times, an application requires the functionality of an operating system, such as file storage, embedded databases or sophisticated device control. A general purpose operating system used in an embedded system may not do the trick. Additionally, it may not be possible to develop the application development on the host due to lack of hardware or support tools.
You can simplify the design of a typical embedded system by using a DOS PC for application development and later linking to your DOS to create an application ROM. You can debug the application on the host PC and define all the DOS entry points used. Later, you can either replace the DOS functionality or utilize an embedded operating system with custom device drivers for your unique hardware. The embedded operating system approach eliminates work and becomes a tool for the developer. Additionally, the embedded operating system becomes an abstraction layer between the application and the embedded system hardware.
Non-Intel Applications
When the operating system design is portable, there are new opportunities that open up for both traditional and embedded applications. One possibility in the area of embedded systems is to design your system and develop it as you would for a normal PC embedded system, then use cross-development tools to recompile the application. When combined with the cross-compiled operating system, software verification reduces to the level of regression testing on the new hardware.
Another reason for developing your own operating system is simply to adapt it to a new platform. There are many non-Intel processor single board computers. To approach a company such as Microsoft to adapt MS-DOS to this custom hardware would be both time and cost prohibitive. However with an operating system designed to be portable, moving the operating system to a new target is a question of cross development tools.
DOS Clone
Another way to view DOS-C is as a declaration of independence. As the desktop computer market matures, Microsoft is moving away from MS-DOS as the foundation of its GUI products. Another example of the trend away from DOS is the decreasing size of the DOS clone market. Products such as DR-DOS are rapidly disappearing from the scene. This leaves older applications in an orphan state. Creating your own version of DOS solves this problem.
Extending DOS
Have you, as a developer, ever looked at the DOS API and felt that if it had just one additional feature, your professional life would be easier? Taking the challenge head-on and creating your own version of DOS is an excellent way of extending DOS.
A typical technique of extending DOS is the Novell approach. When they created their network product, it was as an extension to the existing MS-DOS INT 21h call. They captured the call and looked for their own system calls. If it happened to be one, the network product would service it. Otherwise, it would restore the processor to its original state and pass the call through to DOS. Microsoft also adopted this interrupt chaining with its standard INT 2fh multiplex interrupt chain in version 3.X and above.
With your own DOS, you can extend you system calls to cover your own special call by simply adding it to the function call handler. You write your extension, test it and the integrate it into your own DOS. The simplicity behind this type of customization gives you, the developer, enormous power over your applications environment.