Is Anyone Seriously Trying To Get A Full Windows Kernel Running?, seriously, i'd do it myself if i were legally allowed Options

[torne]

Yes, I have read the stickies, and other long tedious posts filled with criticism.

It would be amusing to see a full Windows boot on the Xbox, and there doesn't seem to be that much to do tbh: poke the kernel/HAL/etc to not touch the bits of the address space and PCI bus that kill the machine, and then write a couple of drivers (graphics, mostly, much of the rest of the hardware should be supported). The latter is pretty easy; the Linux port's drivers are there as a reference for how to program the hardware.

Is anyone seriously looking at fixing the kernel/HAL to avoid the hardware bugs? It's only a couple of megs of code, with full symbolic information available...
I'd do it myself but I have access to, and have spent a lot of time working with, the Windows source (legally under signed NDA) and thus it's extremely dubious for me to reverse-engineer the Windows kernel - I am 'tainted' and would get the project in legal difficulties =(

If the fixes to the existing binaries are done I'd be interested to help write the few required drivers, which would not be subject to such restrictions..

[x-fox]

Well if you are saying you have access to the windows kernal source, I have a few questions for you.

Which specific windows kernal build do you have access to?

What specific patches would apply to the kernal to make it stable enough to run on the xbox hardware?

What bios would you use bearing in mind a standard pc bios would not work?

What graphics driver would you use bearing in mind that the nvidia source has never been released?

Seriously if you cannot provide meaningful answers to the above there would be no point continuing with this thread. Its all been said before.

[torne]

Well if you are saying you have access to the windows kernal source, I have a few questions for you.

Sure.

Which specific windows kernal build do you have access to?

Windows XP SP1, no post-SP1 patches. I forget the build number as I don't have access to it from this computer. I have access to the entire Windows Shared Source distribution, not just the kernel - everything from HAL to device drivers to Explorer and so on, though I've never worked outside of the core code (kernel, hal, devices, NTDLL, etc).

What specific patches would apply to the kernal to make it stable enough to run on the xbox hardware?

Based on what I've read about the xbox so far, and looking at the changes required for xbox-linux, it would be neccecary to amend which parts of the memory and IO map are probed for devices (disabling support for finding SuperIO chips and other legacy controllers, since it appears that some of those probe locations cause the box to lock), and to amend which parts of the PCI bus space are probed (as probing 0.00.1 or 0.00.2 cause crashes). The only other issues I'm aware of from the Linux code and from general discussion is the eject fix to stop the box resetting on DVD eject, and the nonstandard poweroff/reboot sequence, but those could both be left until later.

What bios would you use bearing in mind a standard pc bios would not work?

You don't need a BIOS - only NTLDR requires BIOS services, and very few at that - it gets into protected mode as soon as possible. Someone would have to write a new loader to get the kernel, HAL, disk driver, registry and the few other bits that NTLDR is normally responsible for loaded, and appropriate virtual address space mappings set up, but I've done that before so it must be possible. The entire process is well-documented by publically available books on Windows internals.

On non-legacy hardware the kernel needs no BIOS services and simply accesses devices via the PCI space. No sensible modern OS uses the BIOS for anything other than first-stage bootloading and perhaps initial hardware detection in the early boot.

What graphics driver would you use bearing in mind that the nvidia source has never been released?

I'd get the required details from the open source 'nv' driver for X.org under Linux, and write a new driver from scratch based on nv as documentation. It doesn't support 3D but it can bring up a 2D framebuffer on the xbox just fine, with no binaries or source from nvidia (all reverse engineered).

Seriously if you cannot provide meaningful answers to the above there would be no point continuing with this thread. Its all been said before.

Meaningful enough for you? I can't answer any specific questions on any unpublished details of the Windows code, but anything else is fair game.

You sound like you think you know what you're talking about.. I assume you've implemented device drivers and other kernel-mode code in Windows before? Or have at least read MS Windows Internals?



As for you, "nt authority", suggesting doing anything based on the leaked code is a relatively bad plan, plus you don't seem to know an awful lot about what would actually be required...

[nt authority]

As for you, "nt authority", suggesting doing anything based on the leaked code is a relatively bad plan, plus you don't seem to know an awful lot about what would actually be required...

I take your constructive criticisim onboard torne and admit I dont know an awful lot about this subject yet am willing to learn; I furthermore acknowledge that it appears you do know an awful lot (compared to most other persons in this forum anyway) and I appreciate your input as it is the most logical approach I have seen in a thread on this site. By the way, my expertise is not in code but in neuroscience, cognition, and artificial intelligence.

I am currently working within a team of consultants with a Private Corporation who require massive computing power for processing specific tasks I can not mention as this company's client is a Government Department and, as I am under a NDA with secrecy provisions, doing such would be breaking the law.

The construction of a cluster has been made a sub-project and we are considering using the XBOX conoles as nodes for this cluster and are looking for persons willing to assist in the creation of a true and proper and complete XBOX native Windows operating system. If persons are detected in this forum to have sufficient knowledge and are interested we are prepared to negotiate to employ them under contract.

------------------------------------------------------------------------------------------------------------------------------
This Windows-XBOX-OS would be created by the exact methods torne has stated:
modified NTLDR/OSLOADER.EXE, NTDETECT.COM, NTOSKRNL.EXE, HAL.DLL, NTDLL.DLL etc to avoid the system being locked and drivers specific for the XBOX.
-----------------------------------------------------------------------------------------------------------------------------

The only further modification would be the inclusion of project specific APIs that are required by my program's end result: these include Message Passing Interface, Terminal Services Device Redirector, and Vitual SMP code written into the NTOSKRNL itself so that such is executed within privileged RING 0. This code is required to make all the cluster salve node XBOX's devices (CPU, HDD, RAM, DVD, etc) "divert" transparently to the primary domain controlling server. Ultimately we want the end machine to seemlessly "see" these devices as being local and for the server to have full ownership. Using 9 XBOX slaves for example, and 1 Windows 2003 Compute Cluster Server/Windows 2000 ADS, the Server Machine would see it had 10 CPUs (Virtual SMP code into kernel) as well as 10 DVD drives and 10 HDDs etc... I am presuming a combination of MPI,SMP,and Device Redirection code written into NTOSKRNL could allow such to be achieved by initing these as described during phase 0.
------------------------------------------------------------------------------------------------------------------------------
May I please ask torne why do you state using Windows leaked source would be a bad plan -
are you stating this from a perspective of computer science or law and ethics ?

furthermore,

what made you state that it appears i dont know what needs to be done to start ?

am i misunderstanding your concept of modifying the NTLDR, NTOSKRNL, and HAL to make windows boot ?

did you understand the WinCE-OS as "superBIOS-on-LPC" concept was to act as nothing more than an extended firmware device providing a layer to STOP windows' PCI scans reaching the XBOX hardware and to report back to the LDR/KRNL/HAL that such scans were completed ok ?

is this not possible - to use an intermediate layer (winceOS or even XBE application) to "trick" windows into moving on past the PCI scan phases through a null emulation as such ?

[torne]

I am currently working within a team of consultants with a Private Corporation who require massive computing power for processing specific tasks I can not mention as this company's client is a Government Department and, as I am under a NDA with secrecy provisions, doing such would be breaking the law.

Then sign a Shared Source agreement with MS and build your own Windows from the real code. Seriously dude, you wouldn't need help from some random xbox forum if that were the case

May I please ask torne why do you state using Windows leaked source would be a bad plan -
are you stating this from a perspective of computer science or law and ethics ?

I have been told by various people that the source is not complete and does not correspond to a stable build; I have intentionally avoided looking at the leaked code for legal reasons. Creating a derived work of Windows based on an illegal source code distribution is an invitation to get sued really hard, especially on a public forum

did you understand the WinCE-OS as "superBIOS-on-LPC" concept was to act as nothing more than an extended firmware device providing a layer to STOP windows' PCI scans reaching the XBOX hardware and to report back to the LDR/KRNL/HAL that such scans were completed ok ?

You can't do that. PCI bus probing is done by Windows mapping particular physical addresses (the PCI controller's mapping regions) and writing/reading them. This does not go through any BIOS or intermediate layer; the only way to prevent this reaching the hardware is to modify Windows, or to virtualise the entire machine. Windows does not depend on any BIOS services once NTLDR has proceeded far enough to enter protected mode, which is *very early* in the boot process and thus there is nothing you can substitute, it simply talks to the hardware directly.

The clustering application you talk about would be mind-bogglingly difficult to implement even with a full copy of the Shared Source tree. With binaries or partial leaked source you may as well give up now.

I have installed the MS XBOX SDK into Visual Studio.NET (also with MSDN Universal Subscription tongue.gif) and, for experimental reasons only, have introduced the code from the "leaked" nt 4 source code for the loader subcomponent as found in \nt\private\ntos\boot\

So you have a stolen SDK to compile some stolen source with, good for you

There's no sensible reason to use the Xbox SDK even if you had a legal copy, of course; it would be far more sane to take the approach used by Cromwell and have the loader become the OS as soon as possible, rather than depending on the Xbox kernel for functionality - it would probably be easiest to start off without making it an XBE at all, but by making it a BIOS replacement as with the normal Cromwell build - throw the Xbox kernel out completely. Hell, base it on Cromwell.

Building NTLDR is not really first-things-first. It would be more useful to ignore it and to instead devise a prelinking stage run on a development PC, which would take the required files and turn them into a prelinked image (registry and all) ready to run from a particular address under a particular set of virtual address mappings. A tiny stub executable with almost none of the functionality of NTLDR that just installs the correct mappings and hits the entry point would do.

NTLDR assumes all kinds of things are present that just aren't on the Xbox, you are not going to make rapid progress starting from it.

what build commands/options/settings should be used in VS.NET so this does build ?

Windows code is not built by Visual Studio but by an extremely complex set of custom build scripts inside the source tree. I have no idea how you would get it to run from VS.

what modifications should be made to the dirs/makefile/sources so this does compile into an XBE ?

You have no idea, do you. What about "start again from scratch"?

what modifications need to be done so that, when NTOSKRNL.EXE receives control, it is able to do such low level things such as CPU initialization even though the XBOX has already done a processor initialization ?

NTOSKRNL doesn't *do* CPU initialization or anything else 'low level'. Those things are done by the PC BIOS long before NTLDR even takes control, and NTLDR is the one responsible for creating the basic kernel page tables and enabling protected mode.

It's readily apparant that you know nothing about the Windows boot process; I suggest you read the book I linked in my previous post.


Looks like the answer to my question is 'nobody'. Bah.

[x-fox]

Looks like the answer to my question is 'nobody'. Bah.

Of course the answer is nobody. I don't realy think you had to ask the question. Just a flick arround this forum reveals there is nobody with the required tools and most importantly skills (me included) (you maybe excluded) to port windows to the xbox. Sure there has been plenty of ideas, some good (reactos and win98 emulated), some outragiously silly (windows XP with 64MB ), some damn right stupid.

This windows forum has been seriously pruned of late. It doesnt take genious to work out why that is.
Anyway getting back to your question. It does sound like you are the first one who has put forward some good ideas. But it also sounds like you are not prepared to act upon them. So can I ask why you did ask the question. Do you have something in mind for which you are trying to drum up some support?

This post has been edited by x-fox: Feb 25 2006, 06:50 AM

[torne]

This windows forum has been seriously pruned of late. It doesnt take genious to work out why that is.

I guess.

Anyway getting back to your question. It does sound like you are the first one who has put forward some good ideas. But it also sounds like you are not prepared to act upon them. So can I ask why you did ask the question. Do you have something in mind for which you are trying to drum up some support?

I asked because I'd like to help someone do it.

The part of the process I can't actually *do* is to disassemble the kernel and related bits in order to remove the problematic parts that kill the xbox, as mentioned above. I can't do this in good faith because I'm tainted by having seen and worked on the source code for those very components, and thus reverse-engineering them would in all likelihood be considered a violation of the NDA (or at least, it would be hard to prove that I *didn't* use my knowledge of the code to aid the process...)

The rest of the process - getting a bootloader working, implementing device drivers, etc - I am perfectly able to contribute to, as the information required is publically available (on MSDN, published in books, etc).

So, what would be handy would be someone who knows about, and has the tools for, reverse engineering binaries, who knows as much about the Windows internals as possible without having been exposed to the source, who can then do the binary hacking parts
Though, knowing about Windows would probably be optional, since there are plenty of sources I can point to (the above-mentioned book, the DDK, etc) for information, and much of the symbolic information for the binaries is available.

Oh, and who says Windows XP in 64MB is stupid? As has been previously mentioned, Windows is more componentised than most people are aware of (e.g. XP Embedded) and it's possible to exclude an awful lot if you are building a system image from scratch. Using an older kernel (2000?) may be more reasonable, but I'd think it would be easier to port an NT-based kernel than a consumer Windows (98, etc) - NT is much better at platform independance, having been ported to multiple architectures and systems in the past - there are far less assumptions about PC hardware present in the core system.

This post has been edited by torne: Feb 25 2006, 02:03 PM