Recovering deleted files: metadata recovery vs carving

Deleting a file rarely erases its content: the file system marks the metadata and clusters as free, and the bytes stay until the space is reused or the device discards them. This lesson compares metadata recovery and file carving: what each gives back, what limits it, and how to validate and document the result. Exam questions often ask for the exact hash input, record number or byte offset involved in such a recovery.

Two families of recovery

Metadata recovery works while the file system still describes the deleted file. The structure that held the file's name, size and location is marked unused but has not been overwritten, so a tool can follow the old pointers to the old clusters.

• NTFS. On deletion the in-use flag in the MFT record header (the 2-byte flags field at offset 0x16) is cleared: 0x0001 becomes 0x0000 for a file, 0x0003 becomes 0x0002 for a directory. The record's bit in the MFT's own bitmap and the file's cluster bits in $Bitmap are cleared, and the entry is removed from the parent directory's index. The record itself, with its $STANDARD_INFORMATION, $FILE_NAME and $DATA attributes, normally stays intact until the record is reused. A resident file (data inside the record) is recovered from the record alone.
• FAT. The first byte of the 32-byte directory entry is set to 0xE5 and the file's cluster chain in the FAT is zeroed. The rest of the name, the timestamps, the starting cluster and the file size remain, so a tool can recover the file by assuming it was contiguous and reading ceil(size / cluster size) clusters from the starting cluster. On exFAT the in-use bit of each entry type is cleared (0x85 becomes 0x05, 0xC0 becomes 0x40). Lesson 1.5 covers both layouts.
• ext4. Recovery through metadata is much harder. When ext4 deletes a file it typically zeroes the extent entries in the inode and sets its size to zero, so the inode no longer records where the data was. The name may survive in the directory block, and older copies of the inode may survive in the journal (Lesson 1.6), but often carving is the only option.

File carving needs no file system metadata. The examiner searches raw bytes, usually unallocated space or a whole image, for the structure of known file types: a header signature, a footer or other markers. It also works on reformatted volumes, damaged file systems and memory captures.

What each method keeps

Carving loses file system metadata, not metadata inside the content: a carved JPEG can keep EXIF tags such as DateTimeOriginal, and a DOCX its docProps/core.xml. Report these as embedded application metadata, not file system timestamps.

What limits recovery

Reuse and overwriting. Both methods fail when space is reused, but differently. With metadata recovery, the record may survive while its clusters now belong to another file, and the tool will extract the new content under the old name. With carving, a header may survive while later clusters were overwritten, giving a file that starts correctly and then turns into unrelated data. Once an MFT record is reused, its metadata is gone, although the clusters may still be carvable.

TRIM on SSDs. When a file is deleted on an SSD that supports it, the operating system sends a TRIM command (UNMAP on SCSI, Deallocate on NVMe), at once or in a later scheduled pass depending on the OS and its settings, telling the drive which logical blocks no longer hold data. The drive erases those pages during garbage collection, and many SSDs return zeros for trimmed blocks immediately, before any physical erase. An image acquired afterwards usually shows zeros, so neither method can recover the data. Formatting an SSD can trim the whole volume.

USB flash drives. Most USB flash drives and many USB bridges do not support TRIM over USB mass storage, so deleted data on a typical thumb drive generally remains until it is overwritten. This depends on the device and the bridge: some enclosures and UASP bridges pass UNMAP commands through. Do not assume either way; zeros where deleted data should be are consistent with TRIM or wiping, and should be reported as such.

Signatures, footers and structure

Header/footer carving copies from a header to the next matching footer, or up to a maximum size. It is fast but naive. A JPEG with an EXIF thumbnail contains a second FF D8 FF … FF D9 pair, so the first FF D9 may end the thumbnail rather than the photo. A PDF edited with incremental updates contains several %%EOF markers, and the last one ends the file.

Structure-based carving parses the format. A JPEG carver walks the marker segments, each with a 2-byte length, until the image data, then looks for FF D9. A ZIP carver reads the end of central directory record, whose 2-byte comment length at offset 20 gives the exact end of the file, and checks that the central directory entries point back to valid local headers. In Office Open XML files, [Content_Types].xml and folders such as word/ or xl/ identify the document type.

Because non-resident files start at a cluster boundary, carvers often test for headers only at sector or cluster starts, which is faster and reduces false hits.

Fragmentation and validation

Simple carving assumes contiguous storage. A fragmented file is carved as its first fragment followed by whatever came next on disk; a JPEG typically decodes correctly at the top and turns gray or garbled lower down. Every carved file must therefore be validated: open or parse it, check that its internal structure is consistent (ZIP central directory, PDF cross-reference table, PNG chunk CRCs), and compare its hash with known files where available, such as a hash list of files of interest or a known-good set used to exclude system files. A partial file will not match a reference hash even when recognizable; report it as partial.

Tools as examples

• Metadata recovery: The Sleuth Kit fls -d -r -p lists deleted entries with their full paths; icat extracts a file's content by metadata address; blkstat shows whether a cluster is allocated; ifind -d finds which metadata entry owns a cluster. Autopsy wraps the same functions.
• Carving: PhotoRec (structure-aware for many formats), Foremost and Scalpel (configurable header/footer carvers) each write an audit or report file listing where each file was found.
• Feature extraction: bulk_extractor scans raw bytes for features such as email addresses, URLs and domain names, recording the byte offset of each.

If you carve the output of blkls (unallocated clusters only), offsets refer to that stream; map its block addresses back to the volume with blkcalc -u before reporting.

Worked example 1: recovering through the MFT

fls -d -r -p -o 2048 disk.E01 on an NTFS volume image (volume starting at sector 2,048, 4 KiB clusters) lists a deleted file as MFT record 41. The record's flags field reads 00 00. Its non-resident $DATA attribute shows a real size of 14,862 bytes, an allocated size of 16,384 bytes and the run list 31 04 A0 3F 01 00.

1. Decode the run. Header 0x31: the low nibble (1) is the size of the length field and the high nibble (3) the size of the offset field. Length 0x04 = 4 clusters. Offset A0 3F 01 little-endian = 0x013FA0 = 81,824. The data occupied clusters 81,824 to 81,827; the terminating 00 ends the list.
2. Compute offsets. Volume offset: 81,824 × 4,096 = 335,151,104. Image offset: 335,151,104 + 2,048 × 512 = 335,151,104 + 1,048,576 = 336,199,680. The run covers 4 × 4,096 = 16,384 bytes, of which 14,862 are file data and 1,522 are slack.
3. Check $Bitmap. Each cluster has one bit, least significant bit first. 81,824 / 8 = 10,228 remainder 0, so clusters 81,824 to 81,827 are bits 0 to 3 of byte 10,228. If all four bits are 0, the clusters are unallocated. If any is 1, the cluster now belongs to another file: ifind -o 2048 -d 81824 usb.E01 names it, and the content cannot be attributed to record 41. A clear bitmap does not prove the clusters were never reused and freed again, so validate the content.
4. Extract, hash and validate. icat -o 2048 usb.E01 41 > rec41.bin writes exactly 14,862 bytes (do not use -s, which adds slack). Hash it with SHA-256. If the name ends in .pdf, the first bytes at image offset 336,199,680 should be 25 50 44 46 2D, the file should end with %%EOF and it should open in a PDF parser.

Worked example 2: carving a JPEG

A carver reports a JPEG header FF D8 FF E1 at image offset 419,430,400 (sector 819,200). Relative to the volume, (419,430,400 − 1,048,576) / 4,096 = 102,144 exactly, so the header sits at the start of cluster 102,144, as a file start should. The structure-based carver skips the EXIF segment, including its embedded thumbnail, and finds the end-of-image FF D9 at offset 419,478,226. The file length is 419,478,226 + 2 − 419,430,400 = 47,828 bytes. The output file receives a generated name, typically built from a sequence or sector number. Validate it by decoding the whole image, hash it, and record that its original name, path and file system timestamps are unknown.

Documenting recovery

For each recovered file record: the source image and its verified hash; the partition offset; the method (metadata recovery or carving) and the tool and version; for metadata recovery, the MFT record number or directory entry location and the cluster runs; for carving, the starting byte offset and length; the SHA-256 of the recovered file; the validation performed and its result; and, for carved files, an explicit statement that the original name and timestamps are unknown. This supports the repeatability expected by ISO/IEC 27042.

Older copies of deleted files may also exist in Volume Shadow Copies and file system snapshots; Lesson 1.7 covers APFS snapshots, and Volume Shadow Copies are covered later in the course.

Exam focus: Expect to compute a byte offset from a cluster number, decode a short data run, and decide whether a deleted file's clusters have been reused. Know the signatures and end markers in the table, and which properties survive each method. Know that the SHA-256 of recovered content must be computed over the logical size only, without slack, and how to find the MFT record number of a deleted file.

Key points

• Metadata recovery uses surviving file system records and keeps name, path, timestamps and exact size; carving keeps only content and gets a generated name.
• NTFS and FAT deletions usually leave locations intact; ext4 typically zeroes the extents, so carving is often needed.
• Before trusting recovered content, check the allocation bitmap and validate the file by parsing it and hashing it.
• TRIM on SSDs often makes deleted data read back as zeros; USB flash drives generally do not receive TRIM, depending on device and bridge.
• Header/footer carving fails on fragmented files and nested signatures; structure-based carving is more reliable but still needs validation.
• Document offsets, method, tool version, SHA-256 and, for carved files, that the original name and timestamps are unknown.