Jump to content


Photo

Xbpartitioner 1.1 *does Not* Format With 128k Clusters With Drives >


  • Please log in to reply
27 replies to this topic

#1 wildonrio

wildonrio

    X-S Member

  • Members
  • Pip
  • 103 posts

Posted 14 October 2008 - 08:24 PM

Corrected title (sorry, it got truncated): Xbpartitioner 1.1 *does Not* Format With 128k Clusters With Drives > 1 TB

XBPartitioner version 1.1 is not coded to correctly handle clusters. I just tried this with a 1.5TB drive installed (using a SATA-IDE adapter) and all it would do is 64k clusters, which means something is wrong with the code and we need a new release. I think everyone assumed it would work but no one could test it since they didn't have a drive larger than 1 TB to test it with. Here is the problem code:

QUOTE
void SelectClusterSize()
{
g_iClusterSize = 16;
ULONG compare = 0x20000000;
while (PartTbl.TableEntries[g_iCurrentPart].LBASize > compare)
{
compare = compare * 2;
g_iClusterSize = g_iClusterSize * 2;
}
}


With this code, the program will format partitions incorrectly like so:

0GB - 512GB ====> 16k clusters
512GB - 1TB ====> 32k clusters
1TB - 2TB ====> 64k clusters
2TB - 4TB ====> 128k clusters
...and so on

Instead we would like partitions formatted like so:

0GB - 256GB ====> 16k clusters
256GB - 512GB ====> 32k clusters
512GB - 1TB ====> 64k clusters
1TB - 2TB ====> 128k clusters
...and so on

The fix is quite simple. Here is the code once again, but this time with the fix. I have bolded the fix for clarity:

QUOTE
void SelectClusterSize()
{
g_iClusterSize = 16;
ULONG compare = 0x10000000;
while (PartTbl.TableEntries[g_iCurrentPart].LBASize > compare)
{
compare = compare * 2;
g_iClusterSize = g_iClusterSize * 2;
}
}


Notice that "compare" is a hexadecimal number that represents the kilobyte upper limit on each cluster size. If we examine the problem code, we see that "compare" is initially set to 0x20000000. Convert this to decimal and we get 536870912 kilobytes, or 512 gigabytes. So the code is saying that as long as partition is less than or equal to 512 gigabytes, set the cluster size to "g_iClusterSize", which is initially set to 16. This is wrong.

Now, if we change the initialization of compare to 0x10000000 (268435456 kilobytes, or 256 gigabytes), everything works out as it should. Does this make sense to anyone, or am I missing something?

I would love to compile this new code myself to test but I don't have the XDK, and without compiling and debugging it, I can't be 100% certain...

Can someone please recompile this so we can get some bigger drives working in the XBox? Thanks!!

Edited by wildonrio, 14 October 2008 - 08:37 PM.


#2 Bomb Bloke

Bomb Bloke

    X-S Transcendental

  • Head Moderators
  • PipPipPipPipPipPipPipPipPipPip
  • 6,568 posts
  • Gender:Male
  • Location:Tasmania (AU)
  • Xbox Version:v1.0
  • 360 version:none

Posted 15 October 2008 - 04:55 AM

I could never work out exactly what "compare" stood for, but my best bet was that it dealt with the number of entries in the partition's cluster index table. Cluster based corruption is caused by having too many, but the larger the things are the less you need to fill the drive.

Nevertheless I'm pretty sure the code is already correct in terms of the initial starting value I have there. But according to your statements, you find that the cluster size does not change from 16kb to 32kb when you cross the 256gb mark - I find that a little problematic to rationalise, considering that it worked fine in my tests, and people with larger drives then I own have confirmed it correctly jumps to 64kb clusters when you break the 512gb mark.

Can you confirm that XBP 1.1 uses the incorrect cluster size for all partition combinations with your drive? If so, provide some details on your BIOS, drive make, and console version.

If it only stuffs up when crossing the 1TB mark then I reckon there's gotta be an overflow in there somewhere, but I'm not sure where... Heck, I don't even know how many bytes a ULONG takes up these days. Is it still four? Hope so, should be an easy fix if that's the case.

Mind you, I still recommend a split partition setup. To my mind 128kb per cluster is wasteful to the extreme. A 32kb and 64kb cluster split would be vastly more efficient.

There's also the possibility even if XBP does decide that 128kb is the cluster size it's gonna format with, that doesn't mean 128kb clusters will work. You won't know for sure until you get 1tb of data on there. wink.gif

Furthermore, another guy with a 1.5tb drive had trouble after the format was done (even though he went with a split setup). Though maybe I should get him to take a look at this, perhaps it'll offer some insight on his case as well.

#3 emailchaos

emailchaos

    X-S Enthusiast

  • Members
  • 6 posts

Posted 15 October 2008 - 09:16 AM

QUOTE
If it only stuffs up when crossing the 1TB mark then I reckon there's gotta be an overflow in there somewhere, but I'm not sure where... Heck, I don't even know how many bytes a ULONG takes up these days. Is it still four? Hope so, should be an easy fix if that's the case.


Yep, looks like an overflow to me. A ULONG is indeed 4 bytes, and for partitions larger than 1 TB we'd be using 4294967296 (0x100000000). The range of a ULONG is 4294967295. Doh!

Bottom line - I think it should be using a ULONGLONG...


#4 Bomb Bloke

Bomb Bloke

    X-S Transcendental

  • Head Moderators
  • PipPipPipPipPipPipPipPipPipPip
  • 6,568 posts
  • Gender:Male
  • Location:Tasmania (AU)
  • Xbox Version:v1.0
  • 360 version:none

Posted 15 October 2008 - 10:05 AM

So a ULONG is four bytes... thanks for the heads up. smile.gif I assume a ULONGLONG is something like eight?

But, the thing that gets me is this:

If "compare" is overflowing, then the "while" condition should always be true and the program would enter an infinite loop (that is, the X-Box'll crash if you try to define a 1tb+ partition).

If the LBASize function wants to return a ULONG, then there should be no infinite loop but if you try to define a partition 1tb+ the cluster size should drop down to 16kb.

Neither of these things are happening... But yeah, there's certainly room for an overflow error there. huh.gif

#5 wildonrio

wildonrio

    X-S Member

  • Members
  • Pip
  • 103 posts

Posted 15 October 2008 - 05:36 PM

Ok after I did some more intense testing, it looks I was wrong about wrong cluster sizes. It is indeed an overflow. Using xbpartitioner 1.1 I brought the F partition down to 0 GB and start working my way up, the whole time watching in the corner to see what cluster size it would use. In the beginning it said it was going to use 16 KB clusters, and as expected, when I got 256 GB, it switched to 32 KB clusters. When I got to 512 GB, it switched to 64 KB. Finally, when I got 1024 GB, the xbox froze. There is definitely an overflow. What now?

Edited by wildonrio, 15 October 2008 - 05:39 PM.


#6 emailchaos

emailchaos

    X-S Enthusiast

  • Members
  • 6 posts

Posted 15 October 2008 - 08:19 PM

QUOTE
If "compare" is overflowing, then the "while" condition should always be true and the program would enter an infinite loop (that is, the X-Box'll crash if you try to define a 1tb+ partition).

From what wildonrio just mentioned, the xbox does freeze, which confirms the infinite loop, I believe.

QUOTE
If the LBASize function wants to return a ULONG, then there should be no infinite loop but if you try to define a partition 1tb+ the cluster size should drop down to 16kb.

Is the LBASize a ULONG? What is its defined type? If it is a ULONG, then we have another problem, because the LBASize itself could be overflowing...

#7 Bomb Bloke

Bomb Bloke

    X-S Transcendental

  • Head Moderators
  • PipPipPipPipPipPipPipPipPipPip
  • 6,568 posts
  • Gender:Male
  • Location:Tasmania (AU)
  • Xbox Version:v1.0
  • 360 version:none

Posted 16 October 2008 - 12:32 AM

QUOTE(emailchaos @ Oct 16 2008, 03:55 AM) View Post
Is the LBASize a ULONG? What is its defined type? If it is a ULONG, then we have another problem, because the LBASize itself could be overflowing...

Unlikely. "compare" can only increment so long as it's less then LBASize. If LBASize is hitting an overflow, it becomes impossible for "compare" to do the same; hence no infinite loop, and no crash.

Which is all another way of saying "I can't find where LBASize is defined, I think that function is part of the XDK itself and it'd take me too long to hunt it down". wink.gif

#8 emailchaos

emailchaos

    X-S Enthusiast

  • Members
  • 6 posts

Posted 16 October 2008 - 01:00 AM

Ah, yes, that makes sense.

I'm interested in knowing if the overflow fix allows 128k cluster sizes. Let me know! smile.gif

#9 wildonrio

wildonrio

    X-S Member

  • Members
  • Pip
  • 103 posts

Posted 16 October 2008 - 01:38 AM

QUOTE(emailchaos @ Oct 15 2008, 06:36 PM) View Post

Ah, yes, that makes sense.

I'm interested in knowing if the overflow fix allows 128k cluster sizes. Let me know! smile.gif


Ok I used ULONGLONG instead. Good news is, it actually formatted with 128k clusters! Bad news is, the partition is labeled on the dashboard (XBMC) as "Unavailable". As far as FTP, the F partition shows up but nothing will transfer to it without an error. Getting closer! What next?

#10 emailchaos

emailchaos

    X-S Enthusiast

  • Members
  • 6 posts

Posted 16 October 2008 - 01:46 AM

Wow, could be one of several things. Maybe the partition code is not designed to handle this cluster size. Where is the auto-calculated cluster size being used? I tend to think this is the case, if FTP cannot perform a simple file transfer.

If someone can post the code snippet of where the auto-calculated cluster size is used, I will take a look. Hopefully it's not an XDK function, otherwise we'd have to modify the XDK source!

#11 Bomb Bloke

Bomb Bloke

    X-S Transcendental

  • Head Moderators
  • PipPipPipPipPipPipPipPipPipPip
  • 6,568 posts
  • Gender:Male
  • Location:Tasmania (AU)
  • Xbox Version:v1.0
  • 360 version:none

Posted 16 October 2008 - 02:16 AM

I don't think I can do much more for it then that... Back when the initial testing for 1.1 was being done, we had the same issue (as already mentioned).

The formatting command goes along these lines:

QUOTE
status = XapiFormatFATVolumeEx(&partition_str[g_iCurrentPart], g_iClusterSize << 10);


Where "status" is an int (the XBP packs on X-Bins include full source, by the way). Don't think the bitshift is causing an issue here 'cause if it was 64kb clusters wouldn't be working either.

I'm pretty sure that function is bundled somewhere in the XDK library files, but I've no idea as to how to extract them. Plus there's a good possibility that the issue isn't with the code, but rather another limitation of FATX (such as the one that warrants the use of XBP in the first place).

Seems far easier to me to prevent users from defining partitions 1tb+ and call 2tb the final X-Box drive limit. smile.gif

#12 ldotsfan

ldotsfan

    X-S Messiah

  • Dev/Contributor
  • PipPipPipPipPipPipPip
  • 3,100 posts
  • Xbox Version:v1.1
  • 360 version:unknown

Posted 16 October 2008 - 01:47 PM

QUOTE(Bomb Bloke @ Oct 16 2008, 08:08 AM) View Post

Unlikely. "compare" can only increment so long as it's less then LBASize. If LBASize is hitting an overflow, it becomes impossible for "compare" to do the same; hence no infinite loop, and no crash.

Which is all another way of saying "I can't find where LBASize is defined, I think that function is part of the XDK itself and it'd take me too long to hunt it down". wink.gif

LBASize is defined in partition.h

CODE

typedef struct
{
    UCHAR Name[16];
    ULONG Flags;
    ULONG LBAStart;
    ULONG LBASize;
    ULONG Reserved;
} XboxPartitionTableEntry;


#13 emailchaos

emailchaos

    X-S Enthusiast

  • Members
  • 6 posts

Posted 16 October 2008 - 07:13 PM

Now that's interesting. So I wonder where LBASize is being used, and I wonder why LBASize didn't mess up the cluster detection loop, since I would think there would be an overflow problem.

Changing LBASize to a ULONGLONG may have a positive effect...

#14 run088

run088

    X-S Freak

  • Members
  • PipPipPipPipPip
  • 1,003 posts
  • Interests:rebuilding broken xboxes
  • Xbox Version:v1.0
  • 360 version:none

Posted 16 October 2008 - 08:52 PM

For what it is worth I am positive I used 64k on a 1.5tb split between f&g and data corruption occurred after passing 512 gigs.When I first tried the app it froze everytime until I formatted once with an older build on a new install then the new app seemed to work fine.

I am 100% positive I built it correctly so I am positive there is a problem with the 64k partitions.I was going to try 1 big partition with 128k and see if the results were different but you all are saying that you all are having problems there to.



#15 Bomb Bloke

Bomb Bloke

    X-S Transcendental

  • Head Moderators
  • PipPipPipPipPipPipPipPipPipPip
  • 6,568 posts
  • Gender:Male
  • Location:Tasmania (AU)
  • Xbox Version:v1.0
  • 360 version:none

Posted 17 October 2008 - 12:06 AM

QUOTE(emailchaos @ Oct 17 2008, 02:49 AM) View Post
Now that's interesting. So I wonder where LBASize is being used, and I wonder why LBASize didn't mess up the cluster detection loop, since I would think there would be an overflow problem.

Changing LBASize to a ULONGLONG may have a positive effect...

It confuses me too, but if XBP says it's gonna use 128kb clusters and then proceeds throughout the formatting process, then the size detection code has done it's job correctly - nothing more to mess with there.

QUOTE(run088 @ Oct 17 2008, 04:28 AM) View Post
For what it is worth I am positive I used 64k on a 1.5tb split between f&g and data corruption occurred after passing 512 gigs.When I first tried the app it froze everytime until I formatted once with an older build on a new install then the new app seemed to work fine.

I am 100% positive I built it correctly so I am positive there is a problem with the 64k partitions.I was going to try 1 big partition with 128k and see if the results were different but you all are saying that you all are having problems there to.

While it's certainly a problem, I'm not thinking this is related anymore...

My current theory is that XBP 1.1 never formatted your drive. You first formatted with the older build (which used 32kb clusters), then with 1.1 (which froze the first time... and simply didn't work the second time).

Beats me as to why. I suppose the first test would be to try a few different layouts with 1.1, see if the drive layout changes correctly with each consecutive format.

I know 64kb partitions work. Heck, wildonrio here can vouch for that.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users