Standard Archives: Overview

Description

The FS library mounts several standard archives depending on the type of program that has been booted. This document explains the basics about these archives.

ROM Archives

Within the ROM data region of the DS Game Card, all programs store a single directory tree that was created with the makerom tool. The archive that accesses this directory tree is the ROM archive, and is always mounted using the name rom:. The data to include in the ROM can be specified at build time with the makerom tool. This data is arranged in a data structure known as the NitroROM format. For details about how to include data in the ROM, see the reference for the makerom tool.

The ROM archive can be used without any particular issues for normal card applications. With applications that are booted in other modes, however, there is no certainty that a DS Game Card is actually inserted in the console, so the system behaves as though this directory is empty. For information about the various boot types for programs, see the explanation in the OS_GetBootType page.

If you want to use the ROM archive in a program that has started using DS Download Play in the same way that you would with a card application, we recommend using the WFS library. This library is a utility for reconnecting to the parent that originated the download and accessing ROM data via local wireless communication.

In line with the expanded features of the Nintendo DSi system, major changes have been made to operation of ROM archives on TWL-SDK. The changes from the old package, NITRO-SDK, are summarized below.

NAND Archives

The TWL includes built-in NAND memory. In the documentation, the term NAND archive is used as a collective term for several types of archives that are used to access file systems located in this area. The system NAND memory is actually split into several partitions. The save data files stored by each individual NAND application form independent file systems, but they all use the FAT format as their internal structure. See the TWL Programming Manual for more information on the properties of the system NAND memory.

Depending on the NAND application settings, up to 10 archives can be mounted as NAND archives. Each of these will have a different name, such as photo: or dataPub:. For details about each of these archives, see the NAND Application Development Manual.

SD Memory Card Archives

TWL comes with an SD Memory Card slot, and certain program configurations allow this slot to be accessed. The SD Memory Card archive is mounted using the name sdmc:. SD Memory Cards also use the FAT format, so in general they are used in exactly the same way as NAND archives. That said, some care must be taken because there is wide variation in the data transfer speeds of individual SD Memory Cards, and because there is always a chance that the media could be removed or reinserted during access.

Use the FS_RegisterEventHook function to monitor the removal and insertion of media. To get the current state of the slot or verify write-protection, you can obtain the data as root directory information with an operation such as FS_GetPathInfo("sdmc:/", info). However, if the slot is empty or an unknown device is detected, the function will fail and you need to check the details using the value returned by FS_GetArchiveResultCode. (For example, the return value FS_RESULT_MEDIA_NOTHING indicates that the slot is empty.)

In addition, SD Memory Card recordable media are also used with personal computers and many consumer electronic devices. As a result, when using SD Memory Cards, you must always be aware of the potential danger posed by receiving external data whose content has been tampered with. The use of SD Memory Card archives is prohibited, therefore, with one exception. The SD Memory Card archive is convenient for development purposes, so restricted access is allowed for Debug/Release builds in TWL-SDK version 5.1 PR and later.

Memory File Archives

This archive is a nameless archive that exists in the FS library and is not mounted by the system. Nameless files that have been opened by calls to the FS_CreateFileFromMemory function fall under this category of archive.

Because this archive can treat arbitrary memory spaces just as if they were files, this type of archive can be extremely useful if used properly (for example, by storing frequently used small files in memory).

Internal Formats

The various standard archives described above use internal formats that differ from each other in their characteristics. This section describes the basic characteristics of each of these formats.

NitroROM Format

This is a data archive format unique to TWL-SDK that is used by tools such as nnsarc.exe in the TWL-System package as well as the makerom tool. It has the following characteristics.

• With this format, unique IDs are assigned to all files and directories, so calling FS_ConvertPathToFileID repeatedly to get the IDs of all files you will be using in advance allows you to open the files quickly using FS_OpenFileFast thereafter.
• Because this format was designed for ROM storage media, it is difficult to change the structure of a directory tree. Archives that use the NitroROM format do not support operations such as adding, deleting, or moving files or directories.

An overview of the NitroROM format is given in $TwlSDK/docs/technicalNotes/AboutFileSystem.pdf. However, in general developers do not need to be concerned with this format's specifications.

FAT Format

This specifies the FAT file system that is widely used on SD Memory Cards, hard discs, and other such devices. FAT standards are defined in ISO/IEC 9293. There are various publications and specifications that describe the FAT format, so we have omitted a description here, but following are some characteristics to consider when you use it.

• Internal management information is retained for directories and filenames when a file is created, so the amount of disk space required sometimes exceeds the size of the file. Creating a directory requires at least one cluster of available space. The space needed also depends on the internal state of the file system and the length of the path name. If an unknown number of directories and files will be created, be sure to include an adequate margin in your total space estimate.
• There are upper limits on the total number of files and directories that can be created directly under the root directory, and that limit depends on various factors including the FAT format type, the FAT capacity, and the length of the filenames.
• Certain SD Memory Cards may require an extremely long time for the initial access. This time will depend on various factors, including the cluster size and capacity and the manufacturer-specific data transfer speed.
• There are two types of filenames: short filenames (SFNs) and long filenames (LFNs). These types have different character encodings. Long filenames are always stored using UTF-16LE (Unicode), but there is no particular standard for the encoding of short filenames other than the tendency to use the local encoding for the environment in which the file was created. As a result, it is impossible to reliably recognize the character encoding given only a short filename.

The implementation of the FAT driver used with TWL has the following characteristics.

• It supports FAT12/FAT16/FAT32 and SDHC memory cards.
• All archives implemented using the FAT format are processed by a single driver in the ARM7 component, so even if commands are issued simultaneously from multiple threads to multiple archives, those commands are processed one by one.
• The driver exists in the ARM7 component, so the reading and writing of files is affected by the cache line 32-byte boundary.
  If the buffer or transfer sizes specified in ReadFile/WriteFile does not align with the cache line, a memory copy takes place via the temporary buffer inside the library. Asynchronous calls are ignored, and the process always blocks, causing the transfer speeds to decline.
• When creating files or directories, the driver always creates long filenames, regardless of whether the entry name can be represented in 8.3 format.
• If the seek pointer is not 4-byte aligned when reading from or writing to opened files, the access performance drops.
• If a system shutdown is started, all commands that might require a long processing time return FS_RESULT_CANCELED and are canceled. Examples of such operations include the initial access to an SD Memory Card, or batch reads or writes of 2 MB or greater.
  Also, if this occurs with NAND archives, the NAND enters the state that returns FS_RESULT_BAD_FORMAT.
• It internally emulates the read/write lifetime of system NAND memory to achieve conditions that are closer to those of the retail product. Random latency is added to the read access times for Debug or Release builds.
• The directory entries obtained with FS_ReadDirectory are not sorted in any particular order; they are passed in the same order in which they are stored in the file system. Special entries (system nodes such as "." or "..", volume labels for SD Memory Cards, and so on) are also not filtered and can be obtained unchanged.
• Due to the driver design, only the date/time modified can be obtained using FS_ReadDirectory (the creation and last-accessed dates/times cannot be obtained). If this information is required, it must be obtained separately by calling FS_GetPathInfo.
• When you create a new filename that is long and not in the 8.3 format, it may require a very large amount of processing time to assign a short filename.
  For example, if you create a file with the name aaaaaaaa101.bin, the first candidate for the shortened name is AAAAAA~1.BIN. Suppose, however, a series of 100 files with the names aaaaaaaa001.bin through aaaaaaaa100.bin already exist in the folder. The new shortened candidate name is recursively compared against all the existing shortened names, from AAAAAA~1.BIN through AAAA~100.BIN. The process would repeat 100 times before AAAA~101.BIN is assigned.
  The naming process is faster if you use the 8.3 naming format for files with sequentially numbered prefixes and suffixes.

A fatal timing error exists that will cause inconsistency in the FAT internal state if an SD Card is removed while it is being accessed. The following describes the malfunction that can occur and a countermeasure.

Operational Comparison of Each Format

The FS library provides functions for manipulating most file systems, and while the various archives are required to be compliant with that interface, it is permissible not to implement operations that cannot be supported due to the nature of the file system. If functions are called for archives of this sort, the error FS_RESULT_UNSUPPORTED is always returned. The support status of each format in relation to main functions is shown in the table below.

List of Error Codes

The error codes defined by the FS library are incorporated into the internal processing of the archives. A number of the error codes are only returned by certain archives. The following table shows the error codes that can be returned from each archive.

See Also

FS Function List


Revision History

2010/07/10 Added note that if the power is disconnected during a command that takes a long time to process, the NAND enters a state that returns FS_RESULT_BAD_FORMAT.
2009/06/08 Added a note about the TWL-specific region and TWL-specific file.
2009/05/13 Explained how to detect insertion/removal of SD Memory Cards and how to determine the state of the slot.
2009/04/13 Corrected document title of the reference material. Added explanation of ROM archive operations in TWL mode.
2009/04/10 Replaced symbols used in the tables with standard notation.
2009/02/27 Added text about the file access behavior of the FAT driver.
2009/01/13 Mentioned that restricted access to SD Memory Card archives is permitted, but only for the Debug and Release builds.
2008/12/08 Added a list of error codes returned by the archives.
2008/10/28 Added note about creating series of files with FAT driver.
2008/10/20 Added mention that ROM verification process is not executed when TLF is loaded.
2008/09/30 Added text about FAT state inconsistencies when SD Memory Card is inserted or removed.
2008/09/26 Added text about insertion and removal of SD Memory cards and the speed of ROM archive access.
2008/09/16 Added text about the FAT driver specification.
2008/09/12 Initial version.

CONFIDENTIAL