COEN 252 Computer Forensics

NTFS

NTFS replaces FAT as the basic file system. With NTFS, there is a much stronger separation between file system and physical storage device, allowing dynamic and basic disks. Currently, NTFS works with MBR disks are GPT disks (for 64b computing). This module describes how to extract data from an NTFS file system mounted on a basic MBR disk.

MBR and GPT Disks

With the introduction of 64b computing with the Itanium processor in the Windows environment, the layout of the basic disk has to be changed to take advantage of the processor architecture. For compatibility purposes, both traditional and the new GPT disks start out with a MBR located in the first sector. The first part of the MBR is the master boot code, that scans the partition table for the active partition, finds the starting sector of the active parition, loads a copy of the boot sector from the active partition into memory, and transfers control to the executable code in that boot sector.

A GPT disk starts also with a MBR. However, it contains only one partition entry with a Partition Table entry of 0xEE. The entry is completely determined: 00 | 00 02 00 | EE | FF FF FF | 01 00 00 00 | FF FF FF FF. Thus, the boot process will go the next consecutive sector (LBA 2) and continue the boot process there.

NTFS Architecture for x86-based Systems


Figure 1: NTSF Architecture.

Figure 1 show how an application communicates with a disk. All file systems operations are done in kernel mode by the system file drivers for NTFS. These use the MBR and the boot sector in order to find their way on the disk.

An NTFS partition itself has a simple structure. At the beginning is the single NTFS boot sector. Somewhere in the partition are the two mirrors of the Master File Table. The Master File Table (MFT) has very large entries. Indeed, many small files are entirely stored within the MFT. Typically, the MFT is allocated about 12.5% of the partition size, though this value can be changed in order to accomodate a file system with untypically many big or untypically small files. If needed, part of this allocation is used to accomodate other files. The remaining portion of large files are stored in the File System Data area, that makes up the bulk of the partition. From this layout, we can see that NTFS is quite a departure from FAT, adapted to the gigantic (by standards of a decade ago) disk sizes and volume of data. A copy of the master file table stores the first four system entries (or of the MFT in order to repair the file system if a disk block turns bad.


Figure 2: NTFS partition structure.

NTSF Boot Sector


Figure 3: NTFS boot sector

The NTSF boot sector (see Figure 3 for an example) has a simple structure. It consists mainly of the boot strap code. The data of greatest important to us is hidden in the BPB and extended BPB. The layout of the boot sector is given in Table 1.

Offset
Length of Field
Meaning
0x00, 0
3B
Jump Instruction
0x03
8B
OEM ID
0x0B
25B
BPB
0x24
48B
Extended BPB
0x54
426B
Bootstrap Code.
0x1 FE
2B
End of Sector Marker

Table 1: NTFS boot sector layout.


Figure 2: Interpretation of NTFS boot sector

The layout of the BPB is presented in the following table.

Offset
Length of Field
Typical Value
Meaning
0x0B,
2B
00 02
Bytes per sector. Typical value is 512B per sector
0x0D
1B
08.
Sectors per Cluster
0x0E
2B
00 00
Reserved Sectors. This value must be 0.
0x10
3B
00 00 00
Reserved. This value must be 0.
0x13
2B
00 00
Reserved. This value must be 0.
0x15
1B
F8
Media descriptor. F8: hard disk; F0: high density floppy.
0x16
2B
00 00
Reserved. This value must be 0.
0x18
2B
3F 00
Not used or checked by NTSF.
0x1A
3B
FF 00
Not used or checked by NTSF.
0x1C
4B
3F 00 00 00
Not used or checked by NTSF.
0x20
4B
00 00 00 00
Reserved. This value must be 0.
0x28
8B
xx xx xx xx xx xx xx xx
Total sectors.
0x30
8B
xx xx xx xx xx xx xx xx
Logical cluster number for the Master File Table (File $MFT)
0x38
8B
xx xx xx xx xx xx xx xx
Logical cluster number for the copy of the Master File Table (File $MFTmir)
0x40
1B
F6
Clusters per MFT Record. If the value is less than 7F, then this number is the clusters per Index Buffer. Otherwise, 2x, with x being the negative of this number, is the size of the file record.
0x41
3B
00 00 00
Not used or checked by NTSF.
0x44
1B
01
Clusters per Index Buffer. If the value is less than 7F, then this number is the clusters per Index Buffer. Otherwise, 2x, with x being the negative of this number, is the size of the file record.
0x45
3B
00 00 00
Not used or checked by NTSF.
0x48
8B
xx xx xx xx xx xx xx xx
Volume serial number.
0x50
4B
00 00 00 00
Not used or checked by NTSF.

Table 2: BPB and extended BPB fields on NTSF volumes

In our example, we identify the records:


Figure 4: BPB and extended BPB

Not all the fields in the BPB and extended BPB are important. We cull the used fields:

MFT and Metadata Files

Because the MFT stores information about itself, its first 16 records are reserved for metadata files, which are used to describe the MFT. They have file names beginning with a dollar sign ($). The remaining records contain the files and folders for each file and folder in the volume:

  1. Master file table $MFT.
  2. Master file table mirror $MftMirr.
  3. Log file $LogFile.
  4. Volume $Volume contains information such as the volume label and the volume version.
  5. Attribute definitions $AttrDef.
  6. The root folder “.”
  7. Cluster bitmap $Bitmap which represents the volume by showing free and unused clusters.
  8. Bot sector $Boot, includes the BPB used to mount the volume and additional bootstrap loader code used if the volume is bootable.
  9. Bad cluster file $BadClus, which contains a list of bad clusters for the volume.
  10. Security file $Secure which contains unique security descriptors for all files within a volume.
  11. Upcase table $Upcase which converts lowercase characters to matching Unicode uppercase characters.
  12. NTFS extension file $Extend, that is used for future use.
  13. Reserved
  14. Reserved
  15. Reserved
  16. Reserved

 


Figure 5: MFT record for a small file.

MTF Records

NTFS creates a file record for each file and a folder record for each folder created on an NTFS volume. The MFT includes a separate file record for the MFT itself. These file and folder records are 1 KB each and are stored in the MFT. The attributes of the file are kept in the MFT. Besides file attributes, each file record contains information about the location of the data blocks of the file. Small files are completely contained in an MFT record. An MFT entry is very much like a database record with many optional fields.


Figure 6: MFT record for a small file.

Figure 5 gives an MFT record for a small file or folder. In general, files of up to ~900B are completely stored within the MFT entry. All MFT entries are 1KB long. In general, an MTF entry contains the following parts:

Folder records contain index information. Small folders reside entirely within a MFT record, but larger ones are organized as a B-tree and have pointers to external clusters that contain folder entries that cannot be contained within the MFT.

Offset
Size
OS
Description
0x00 - 0x03
4
Magic Number: "FILE"
0x04-0x05
2
Offset to the update sequence.
0x06-0x07
2
Number of entries in fixup array
0x08-0x0f
8
$LogFile Sequence Number (LSN)
0x10-0x11
2
Sequence number
0x12 - 0x13
2
Hard link count
0x14-0x15
2
Offset to first attribute
0x16 - 0x17
2
Flags: 0x01: record in use, 0x02 directory.
0x18-0x1b
4
Used size of MFT entry
0x1c-0x1f
4
Allocated size of MFT entry.
0x20 - 0x27
8
File reference to the base FILE record
0x28-0x29
2
Next attribute ID
0x2a-0x2b
2
XP
Align to 4B boundary
0x2c-ox2f
4
XP
Number of this MFT record
0x30-0x1000
Attributes and fixup value

Table 3: Data structure of a basic MFT entry

LSN, File Reference Address

Each MFT record is addressed by a 48 bit MFT entry value.The first entry has address 0. Each MFT entry has a 16 bit sequence number that is incremented when the entry is allocated. MFT entry value and sequence number combined yield 64b file reference address.


Figure 7: File Reference Address

If the file in Figure 6 would be deleted and the MFT entry reused for another file, then the sequence number would become 0041 and the new file reference address 0041000000000012. This makes it easier to determine whether the file system is corrupt.

Hard Link Count

The hard link count is the number of directory entries that reference this record.

MFT Entry Attributes


Figure 8: MFT Record Structure Example

MFT records contain various attributes preceeded by an attribute header (Figure 8). Some attributes are resident, that is, contained in the MFT record, others are non-resident and reside in a cluster outside of the MFT.

Type
OS
Name
0x10
$STANDARD_INFORMATION
0x20
$ATTRIBUTE_LIST
0x30
$FILE_NAME
0x40
NT
$VOLUME_VERSION
0x40
2K
$OBJECT_ID
0x50
$SECURITY_DESCRIPTOR
0x60
$VOLUME_NAME
0x70
$VOLUME_INFORMATION
0x80
$DATA
0x90
$INDEX_ROOT
0xA0
$INDEX_ALLOCATION
0xB0
$BITMAP
0xC0
NT
$SYMBOLIC_LINK
0xC0
2K
$REPARSE_POINT
0xD0
$EA_INFORMATION
0xE0
$EA
0xF0
NT
$PROPERTY_SET
0x100
2K
$LOGGED_UTILITY_STREAM

Table 4: List of possible attributes is defined in $AttrDef, but these are the default values.

$PROPERTY_SET, $SYMBOLIC_LINK and $VOLUME_VERSION existed in NTFS v1.2, but weren't used. They no longer exist in NTFS v3.0 (that used by Win2K). Each MFT record has a Standard Header, followed by a list of attributes (in order of ascending Attribute Id) and an end marker. The end marker is just four bytes: 0xFFFFFFFF. [Reference linux-ntfs project]

Attribute Layout

The data structure for the first 16 B of an attribute is the same for resident and non-resident attributes. After that, it differs. This is because non-resident attributes need to describe an arbitrary number of cluster runs, consecutive clusters that they occupy.

Offset
Size
Description
Offset
Size
Description
0x00
4
Attribute Type Identifier (see Table 4)
0x00
4
Attribute Type Identifier (see Table 4)
0x04
4
Length of Attribute (determines the location of next attribute)
0x04
4
Length of Attribute (determines the location of next attribute)
0x08
1
Non-resident flag
0x08
1
Non-resident flag
0x09
1
Lenght of name
0x09
1
Lenght of name
0x0a
2
Offset to name
0x0a
2
Offset to name
0x0c
2
Flags
0x0c
2
Flags
0x0e
2
Attribute Identifier
0x0e
2
Attribute Identifier
0x10
4
Size of content.
0x10
8
Starting Virtual Cluster Number of the runlist.
0x15
2
Offset to content
0x18
8
Ending Virtual Cluster Number of the runlist.
0x20
2
Offset to the runlist
0x22
2
Compression unit size
0x24
4
unused
0x28
8
Allocated size of the attribute content
0x30
8
Actual size of attribute content
0x38
8
Initialized size of the attribute content.

Table 5: Data Structure for resident (left) and non-resident attributes.

The attribute type identifier determines also the layout of the contents. Table 6 for example gives the layout for the Standard_Information Attribute Header and Table 7 the one for the File_Name..

$STANDARD_INFORMATION
Offset Size OS Description
0x00 8   C Time - File Creation
0x08 8   A Time - File Altered
0x10 8   M Time - MFT Changed
0x18 8   R Time - File Read
0x20 4   DOS File Permissions
0x24 4   Maximum Number of Versions
0x28 4   Version Number
0x2C 4   Class Id
0x30 4 2K Owner Id
0x34 4 2K Security Id
0x38 8 2K Quota Charged
0x40 8 2K Update Sequence Number (USN)

Table 6: Layout of the $Standard_Information (0x10) Attribute

$FILE_NAME
Offset Size Description
0x00 8 File Reference to parent directory
0x08 8 File creation time.
0x10 8 File modification time
0x18 8 MFT modification time
0x20 8 File access time.
0x28 8 Allocated size of file
0x30 8 Real size of file
0x38 4 Flags
0x3c 4 Used by EAs and Reparse
0x34 4 Security Id
0x40 1 Filename length in unicode characters
0x41 1 Filename namespace
0x42   File Name in Unicode

Table 7: Layout of the $FILE_NAME (0x30) Attribute

The time values are given in 100 nanoseconds since January 1, 1601, UTC.

NTFS Versions

NTFS has a very flexible layout unlike that of FAT. This makes it easy for MS to change small aspects of the layout. For example, a Windows XPPro formatted NTFS volume puts the $Log file in a different place than a Windows 2000 formatted NTFS volume, resulting in similar file access times than for FAT. This makes an analysis of NTFS much harder than for FAT, and a serious analysis needs to use software tools.

Examples

Reading an MFT record.

Figure 9: MFT Entry

MFT Entry Header

In the figure above, we see the beginning of an MFT record. At the beginning, we find the magic number 46 49 4C 45 or "FILE". (If the entry is unusable, we would find it marked as "BAAD".) The offset to the update sequence is 00 30. As bytes 7 and 8 (03 00) show, the update array has three entries. Bytes 16 and 17 (59 00) contain the sequence number which shows that this entry has been used 89 times. The next two bytes (02 00)give a link count is 2. The first attribute is located at byte offset 0x38 = 56. The flags are 0x 00 03, which means that the entry is used and is a directory value, since 0x 00 03 = 0x 00 02 | 0x 00 01.

MFT entries could be larger than fit into the normal space. In this case, the MFT entry will start in the base MFT record and continued in an extension record. The file reference to the base file entry is 0x 00 00 00 00 00 00 00 00. This means that this is a base record. Where it not so, then this field would contain a reference to the base MFT record.

The next two bytes (0x 09 00) give the next attribute ID. We can therefore expect that there are eight attributes.

The fixup array starts at offset 0x30. The first two bytes (0x 8c 06) are the last two bytes in every sector of the record. The real last couple of bytes in all the sectors are stored in the fixup array that follows, namely all zeroes. The idea behind the fixup array is to recognize clusters that have been partially written.

Attributes

As we have seen, the first attribute is located at offset 0x38. The attribute identifier is 0x 00 00 00 10, that is, the first attribute is the standard information. The attribute is 0x60 B long. It is resident, and its contents have size 0x48, located at offset 0x18. (Notice that this gives us a sanity check: 0x48 + 0x18 = 0x60.)

Figure 10: MFT Entry First Attribute

We interpret the contents of this attribute according to Table 6. For example, we find that the creation time is given as 0x 22 3d ca 9d ca bb c4 01, which we interpret as an NTFS time. To manually convert NTFS and other time date formats, use . Programmatically, you can use jsourcery's code

    public static long filetimeToMillis(long filetime) {

        filetime -= 116444736000000000L;

        if (filetime < 0) {
            filetime = -1 - ((-filetime - 1) / 10000);
        } else  {
            filetime = filetime / 10000;
        }
        return filetime;
    }
   public static Date filetimeToDate(long filetime) {
        return new Date(filetimeToMillis(filetime));
    }
}

but you need to be careful about converting between little and big Endian. The code basically converts the time to have the January 1, 1970 UNIX zero-point and then calls unix tools. Peter Bergstrom's driver shows how to use it after conversion:

import java.util.*;
class timeStampConverter
{
   public static long filetimeToMillis(long filetime)
   {
     filetime -= 116444736000000000L;
     if (filetime < 0)
     {
       filetime = -1 - ((-filetime - 1) / 10000);
     }
     else
     {
       filetime = filetime / 10000;
     }
     return filetime;
   }

   public static Date filetimeToDate(long filetime)
   {
     return new Date(filetimeToMillis(filetime));
   }	

   public static void main(String args[])
   {
     long time = 127541957167518016L; // Specify your long here!!
     Date timeDate = filetimeToDate(time);			
     System.out.println(timeDate.toString());
   }
}

Figure 10: MFT Entry Second Attribute

The next attribute is the file name (0x 30 00 00 00). As we can see from the second entry, the attribute takes up 0x 70 bytes. The contents start at offset 0x18. At offset 0x42 in the content, we find the file name in unicode, i.e. "COEN35~". The first 8 bytes of the content 0x 0a e4 00 00 00 00 03 00 are the reference to the parent directory, where the upper two bytes are the sequence number and the lower siz bytes are the MFT entry, here 0xe40a. The file sizes are clearly not accurate, as is frequently the case.

Figure 11: MFT Entry ThirdAttribute

The next entry is another file name entry, this time for the long entry.

Since this is a directory, the contents of the directory are organized in a B-tree. To take the structure apart is too tedious for us. More detail are in the book by Brian CarrieBrian Carrierr: File System Forensic Analysis. Other hints about going further can be found at the Linux-NTFS project under sourceforge.

2007 Thomas Schwarz, S.J., COEN, SCU SCU COEN COEN252 T. Schwarz