Inhaltszusammenfassung zur Seite Nr. 1
StorNext
File System Tuning Guide File System Tuning Guide
®
StorNext 3.1.4.1
6-01376-15 Rev A
Inhaltszusammenfassung zur Seite Nr. 2
Document Title, 6-01376-15 Rev A, May 2010, Product of USA. Quantum Corporation provides this publication “as is” without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability or fitness for a particular purpose. Quantum Corporation may revise this publication from time to time without notice. COPYRIGHT STATEMENT © 2010 Quantum Corporation. All rights reserved. Your right to copy this manual is limited by copyright law. Making c
Inhaltszusammenfassung zur Seite Nr. 3
Contents The Underlying Storage System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 File Size Mix and Application I/O Characteristics . . . . . . . . . . . . . . . . . . 5 SNFS and Virus Checking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 The Metadata Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 The Metadata Controller System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 The Distributed LAN (Di
Inhaltszusammenfassung zur Seite Nr. 4
Contents iv StorNext File System Tuning Guide
Inhaltszusammenfassung zur Seite Nr. 5
StorNext File System Tuning The StorNext File System (SNFS) provides extremely high performance for widely varying scenarios. Many factors determine the level of performance you will realize. In particular, the performance characteristics of the underlying storage system are the most critical factors. However, other components such as the Metadata Network and MDC systems also have a significant effect on performance. Furthermore, file size mix and application I/O characteristics may also
Inhaltszusammenfassung zur Seite Nr. 6
StorNext File System Tuning The Underlying Storage System RAID Cache The single most important RAID tuning component is the cache configuration. This is particularly true for small I/O operations. Configuration Contemporary RAID systems such as the EMC CX series and the various Engenio systems provide excellent small I/O performance with properly tuned caching. So, for the best general purpose performance characteristics, it is crucial to utilize the RAID system caching as fully as possibl
Inhaltszusammenfassung zur Seite Nr. 7
StorNext File System Tuning The Underlying Storage System operations involve a very high rate of small writes to the metadata disk, so disk latency is the critical performance factor. Write-back caching can be an effective approach to minimizing I/O latency and optimizing metadata operations throughput. This is easily observed in the hourly File System Manager (FSM) statistics reports in the cvlog file. For example, here is a message line from the cvlog file: PIO HiPriWr SUMMARY SnmsMetaDis
Inhaltszusammenfassung zur Seite Nr. 8
StorNext File System Tuning The Underlying Storage System While read-ahead caching improves sequential read performance, it does not help highly transactional performance. Furthermore, some SNFS customers actually observe maximum large sequential read throughput by disabling caching. While disabling read-ahead is beneficial in these unusual cases, it severely degrades typical scenarios. Therefore, it is unsuitable for most environments. RAID Level, Segment Configuration settings such as RAI
Inhaltszusammenfassung zur Seite Nr. 9
StorNext File System Tuning File Size Mix and Application I/O Characteristics It can be useful to use a tool such as lmdd to help determine the storage system performance characteristics and choose optimal settings. For example, varying the stripe size and running lmdd with a range of I/O sizes might be useful to determine an optimal stripe size multiple to configure the SNFS StripeBreadth. Some storage vendors now provide RAID6 capability for improved reliability over RAID5. This may be pa
Inhaltszusammenfassung zur Seite Nr. 10
StorNext File System Tuning File Size Mix and Application I/O Characteristics and auto_dma_write_length, described in the Mount Command Options on page 19. Buffer Cache Reads and writes that aren't well-formed utilize the SNFS buffer cache. This also includes NFS or CIFS-based traffic because the NFS and CIFS daemons defeat well-formed I/Os issued by the application. There are several configuration parameters that affect buffer cache performance. The most critical is the RAID cache configu
Inhaltszusammenfassung zur Seite Nr. 11
StorNext File System Tuning SNFS and Virus Checking If performance requirements cannot be achieved with NFS or CIFS, consider using a StorNext Distributed LAN client or fibre-channel attached client. It can be useful to use a tool such as netperf to help verify network performance characteristics. SNFS and Virus Checking Virus-checking software can severely degrade the performance of any file system, including SNFS. If you have anti-virus software running on a Windows Server 2003 or Windo
Inhaltszusammenfassung zur Seite Nr. 12
StorNext File System Tuning The Metadata Controller System ethtool tool can be very useful to investigate and adjust speed/duplex settings. It can be useful to use a tool like netperf to help verify the Metadata Network performance characteristics. For example, if netperf -t TCP_RR reports less than 15,000 transactions per second capacity, a performance penalty may be incurred. You can also use the netstat tool to identify tcp retransmissions impacting performance. The cvadmin “latency-tes
Inhaltszusammenfassung zur Seite Nr. 13
StorNext File System Tuning The Metadata Controller System used to realize performance gains from increased memory: BufferCacheSize, InodeCacheSize, and ThreadPoolSize. However, it is critical that the MDC system have enough physical memory available to ensure that the FSM process doesn’t get swapped out. Otherwise, severe performance degradation and system instability can result. The operating system on the metadata controller must always be run in U.S. English. FSM Configuration File The
Inhaltszusammenfassung zur Seite Nr. 14
StorNext File System Tuning The Metadata Controller System StripeBreadth 256K MultiPathMethod Rotate Node CvfsDisk6 0 Node CvfsDisk7 1 [StripeGroup MetaFiles] Status UP MetaData Yes Journal No Exclusive Yes Read Enabled Write Enabled StripeBreadth 256K MultiPathMethod Rotate Node CvfsDisk0 0 [StripeGroup JournFiles] Status UP Journal Yes MetaData No Exclusive Yes Read Enabled Write Enabled StripeBreadth 256K MultiPathMethod Rotate Node CvfsDisk1 0 Affinities Affinities
Inhaltszusammenfassung zur Seite Nr. 15
StorNext File System Tuning The Metadata Controller System Affinity AudFiles ##for Audio Files Only## Read Enabled Write Enabled StripeBreadth 1M MultiPathMethod Rotate Node CvfsDisk4 0 Node CvfsDisk5 1 Note: Affinity names cannot be longer than eight characters. StripeBreadth This setting must match the RAID stripe size or be a multiple of the RAID stripe size. Matching the RAID stripe size is usually the most optimal setting. However, depending on the RAID performance characteristics a
Inhaltszusammenfassung zur Seite Nr. 16
StorNext File System Tuning The Metadata Controller System BufferCacheSize This setting consumes up to 2X bytes of memory times the number specified. Increasing this value can reduce latency of any metadata operation by performing a hot cache access to directory blocks, inode information, and other metadata info. This is about 10 to 1000 times faster than I/O. It is especially important to increase this setting if metadata I/O latency is high, (for example, more than 2ms average latency).
Inhaltszusammenfassung zur Seite Nr. 17
StorNext File System Tuning The Metadata Controller System Example: ThreadPoolSize 32 # default 16, 512 KB memory per thread ForcestripeAlignment This setting should always be set to Yes. This is critical if the largest StripeBreadth defined is greater than 1MB. Note that this setting is not adjustable after initial file system creation. Example: ForcestripeAlignment Yes FsBlockSize The FsBlockSize (FSB), metadata disk size, and JournalSize settings all work together. For example, th
Inhaltszusammenfassung zur Seite Nr. 18
StorNext File System Tuning The Metadata Controller System Average No. of Files Per File System SIze: Less File System Size: 10TB Directory Than 10TB or Larger Less than 10 FSB: 16KB FSB: 64KB Metadata: 32 GB per 1M Metadata: 128 GB per files 1M files 10-100 FSB: 16KB FSB: 64KB Metadata: 8 GB per 1M Metadata: 32 GB per 1M files files 100-1000 FSB: 64KB FSB: 64KB Metadata: 8 GB per 1M Metadata: 8 GB per 1M files files 1000 + FSB: 64KB FSB: 64KB Metadata: 4 GB per 1M Metadata: 4 GB per 1M fi
Inhaltszusammenfassung zur Seite Nr. 19
StorNext File System Tuning The Metadata Controller System This setting is adjustable using the cvupdatefs utility. For more information, see the cvupdatefs man page. Example: JournalSize 16M SNFS Tools The snfsdefrag tool is very useful to identify and correct file extent fragmentation. Reducing extent fragmentation can be very beneficial for performance. You can use this utility to determine whether files are fragmented, and if so, fix them. If your files are prone to fragment
Inhaltszusammenfassung zur Seite Nr. 20
StorNext File System Tuning The Metadata Controller System •Large value for FSM threads SUMMARY max busy indicates the FSM configuration setting ThreadPoolSize is insufficient. • Extremely high values for FSM cache SUMMARY inode lookups, TKN SUMMARY TokenRequestV3, or TKN SUMMARY TokenReqAlloc might indicate excessive file fragmentation. If so, the snfsdefrag utility can be used to fix the fragmented files. •The VOP and TKN summary statistics of the form count avg/q+e min/q+e max/q+e show m
