Saturday, December 22, 2007
Prima Gets A Memory Manager
There's an underlying theme that I have yet to write about. No, it's not the memory manager. We'll get to that shortly. What I'm talking about here is KISS. For those unfamiliar with the acronym, it stands for Keep It Simple, Stupid. While it may be taken as a slight, it's really good advice. When given choices of solutions, choose the simpler. It's easier to code and debug. It's easier to maintain. It's faster to develop, and so on. Unfortunately, we seem to forget this, and create huge, difficult to understand and maintain programs, especially in the operating system space.
A good example of this is Linux. I know this is heresy, but how many people really understand the current kernel? I haven't looked at it since 2.6.10, and I know that there have been changes, so that I can't make the claim unless I go back and spend several hours examining the changes. I'd like to avoid it here. I know that if this operating system gains any popularity whatsoever, it will be difficult to avoid bloat, but I'm going to try. Now, maybe, you'll understand the choices behind the memory management.
Prima's Memory Management Design
Previously, we examined the region abstraction that we'll use to encapsulate the underlying hardware details. We haven't looked at the bigger picture, which is managing the processor's memory space in general. For this, we'll fall back to DOS. While there are many people who will pooh pooh this, the methodology is a simple linked list management system. A processor page of 16 bytes is used, thanks to the relationship of segments to memory, and it can be allocated and freed with a minimal amount of code. Seems to me that it meets all the requirements of a KISS design.
DOS-C/FreeDOS isn't the only operating system to use a linked list for free memory management. There's a similar system used in Minix and others. For Prima, we'll use the original DOS-C code, with some changes. For example, the original code used paragraph offsets. This was by design, and made to match the MS-DOS design. I did not make the original design decisions, but the use of a 16 byte paragraph offset allows the os designers to perform simple increments and decrements of the segment portion of the far pointer to move about memory in a rapid fashion. For Prima, a similar choice should be a 4096 byte page. This will help keep memory manipulation in check with smaller size integers for use in computations. It also allows a simpler migration to paging for future releases. That size should also be configurable for future ports to other platforms.
Another adaptation needed to my original code is to the functions and macros designed to adjust the segment:offset addresses to C far pointers. Since we no longer work in a segmented architecture, that is no longer necessary. The reference type will take care of the physical/logical conversion and there is no far attribute in C outside real mode 8086 compilers. What is necessary is a group of support functions to move data to/from user space and between different users spaces. These functions are in assembler and reside in lowcore.asm. Between this and the free space linked list, we should have everything we need to manage our physical memory.
What will significantly differ from DOS-C is freeing of memory. In DOS, the processes are also in a linked list, and freeing process memory is as simple as acquiring that pointer and calling the correct routine to free memory. That can't happen here because of our region abstraction. Instead, we'll pass the region handle back to the region code and expect back a linked list of deallocated memory. We'll then go through the list and free each piece of the list. This is necessary because when we move to paging, the logical, linear memory space may be highly fragmented pages. We cannot assume that the pages will be consecutive. So we must ask the region management code to convert it for us so we can then manage the free space normally.
Final Notes
What I did not mention is that a rudimentary process structure will be necessary to continue this work. I need it so that I can test the region allocation and deallocation code. I also need to create a user space to move data to and from in order to test the support code. I have some ideas, but I'll reserve that design for a later discussion.
You may also be interested in the current status of the code. I just completed converting the thread code from kernel threads to a more generic threading mechanism that will work for kernel and user space. I've done some testing on the code, but without a user space I cannot test the user space aspect of it. Again, another reason to create a rudimentary process. Alas, a complication typical in many projects. However, the progress I am making is good and I'm looking forward to completing the proof of concept release. I will make full use of the early process code to test ideas for the final code.
A good example of this is Linux. I know this is heresy, but how many people really understand the current kernel? I haven't looked at it since 2.6.10, and I know that there have been changes, so that I can't make the claim unless I go back and spend several hours examining the changes. I'd like to avoid it here. I know that if this operating system gains any popularity whatsoever, it will be difficult to avoid bloat, but I'm going to try. Now, maybe, you'll understand the choices behind the memory management.
Prima's Memory Management Design
Previously, we examined the region abstraction that we'll use to encapsulate the underlying hardware details. We haven't looked at the bigger picture, which is managing the processor's memory space in general. For this, we'll fall back to DOS. While there are many people who will pooh pooh this, the methodology is a simple linked list management system. A processor page of 16 bytes is used, thanks to the relationship of segments to memory, and it can be allocated and freed with a minimal amount of code. Seems to me that it meets all the requirements of a KISS design.
DOS-C/FreeDOS isn't the only operating system to use a linked list for free memory management. There's a similar system used in Minix and others. For Prima, we'll use the original DOS-C code, with some changes. For example, the original code used paragraph offsets. This was by design, and made to match the MS-DOS design. I did not make the original design decisions, but the use of a 16 byte paragraph offset allows the os designers to perform simple increments and decrements of the segment portion of the far pointer to move about memory in a rapid fashion. For Prima, a similar choice should be a 4096 byte page. This will help keep memory manipulation in check with smaller size integers for use in computations. It also allows a simpler migration to paging for future releases. That size should also be configurable for future ports to other platforms.
Another adaptation needed to my original code is to the functions and macros designed to adjust the segment:offset addresses to C far pointers. Since we no longer work in a segmented architecture, that is no longer necessary. The reference type will take care of the physical/logical conversion and there is no far attribute in C outside real mode 8086 compilers. What is necessary is a group of support functions to move data to/from user space and between different users spaces. These functions are in assembler and reside in lowcore.asm. Between this and the free space linked list, we should have everything we need to manage our physical memory.
What will significantly differ from DOS-C is freeing of memory. In DOS, the processes are also in a linked list, and freeing process memory is as simple as acquiring that pointer and calling the correct routine to free memory. That can't happen here because of our region abstraction. Instead, we'll pass the region handle back to the region code and expect back a linked list of deallocated memory. We'll then go through the list and free each piece of the list. This is necessary because when we move to paging, the logical, linear memory space may be highly fragmented pages. We cannot assume that the pages will be consecutive. So we must ask the region management code to convert it for us so we can then manage the free space normally.
Final Notes
What I did not mention is that a rudimentary process structure will be necessary to continue this work. I need it so that I can test the region allocation and deallocation code. I also need to create a user space to move data to and from in order to test the support code. I have some ideas, but I'll reserve that design for a later discussion.
You may also be interested in the current status of the code. I just completed converting the thread code from kernel threads to a more generic threading mechanism that will work for kernel and user space. I've done some testing on the code, but without a user space I cannot test the user space aspect of it. Again, another reason to create a rudimentary process. Alas, a complication typical in many projects. However, the progress I am making is good and I'm looking forward to completing the proof of concept release. I will make full use of the early process code to test ideas for the final code.
# posted by Pat Villani @ 6:09 AM 
Subscribe to Posts [Atom]
Copyright Pasquale J. Villani