Lenovo Storage S3200 with
Fibre Channel
Quick Start Guide
Bryan Reese
David Vestal
Jeff Lin
July 2016
©2016 Lenovo. All rights reserved.
Lenovo Storage S3200 with Fibre Channel
2
Introduction
Reference designs have become an essential part of many customer data centers
allowing for rapid deployment of a system that has been tested and assured to work by a
vendor. They ensure that all of the components work together and can give someone
who is unfamiliar with some aspects of the deployment a starting place to go for
answers. Further, they enable an understanding of where a set of hardware fits into a
data center and allows for a simpler decision process. Why would you evaluate a
solution that does not fit your needs?
This document specifically discusses an infrastructure design using Lenovo Fibre
Channel networking from Brocade, Lenovo servers, Lenovo storage, and Emulex Fibre
Channel Host Bus Adapters (HBAs) by Broadcom. Users who may be using NAS or
iSCSI today and are considering an entry Fibre Channel solution will find the information
contained within this document useful to create a basic configuration.
Once you have completed reading this document, you will understand how to zone a
Fibre Channel switch and how to make LUNs available to your chosen OS for file
storage or OS deployment. It covers all basic steps up until the installation of the
operating system. The document also includes a sizing guide to help understand what
equipment will work best in a given environment.
This design document includes technology and input from three of the leading vendors in
IT today: Lenovo, Emulex (a division of Broadcom Limited), and Brocade. These three
partners have a complimentary portfolio that allows for innovation in an individual and
specialized area while collaborating and providing best of IT solutions. All three vendors
have priorities set for achieving performance that is best in the industry and reducing
deployment times by minimizing administrative complexity.
About the Author
Bryan Reese
Bryan is a solutions architect that works for Brocade on projects that range from SAN to
IP and software to hardware. He has lead projects that involve Docker, Oracle Real
Applications Cluster, and HPC with a focus on how the whole system would operate. In
his career, he has helped customers with large and small deployments find an optimal
solution during both before and after the sales have occurred. His career highlights
include traveling to Germany when he was an IBM employee to work with large super
computers deployed in Garching outside of Munich (LRZ, RZG) and as a Brocade
employee working on engineering problems including creating proof of concepts for
security vulnerabilities such as Shellshock.
David Vestal
Lenovo Storage S3200 with Fibre Channel
3
David is the product marketing manager for Lenovo’s Data Center Group, Enterprise
storage platform. He has over 20 years of professional marketing, sales, and business
development experience across a broad range of telecommunication, banking, and
enterprise IT sectors. Previously he was the product & product marketing manager for
Lenovo's System x Ethernet and Fibre Channel options portfolio.
Jeff Lin
Jeff is a marketing manager for Broadcom Limited at Lenovo, with responsibility for
Emulex Fibre Channel adapters for System x, Flex System, and ThinkServer. He has
over 18 years’ experience in various marketing roles at IBM and Lenovo.
Use Cases
Primarily, the reader of this document will be an administrator or user who desires an
assurance that a design is valid and that all parts have been tested together. This is a
high-level design that may fulfill a purpose perfectly by changing the processor, memory,
or the amount of storage. Details on creating a configuration are in the sizing section of
this document.
The use cases that we have targeted with this design include:
Media streaming
Web hosting
OLTP databases
Hosting communications (IM, email, shared storage)
Private cloud and general usage
Solution Overview
This design is flexible to accommodate differences in environments and computing
requirements. We made optimal selections from the broader product portfolios to ensure
that the target design considers cost, performance, and capacity. The principals
discussed within this document can allow for a small deployment as the primary use
case but can work for a medium sized deployment or even a large deployment when
using pod design philosophies.
Lenovo Storage S3200
Lenovo Storage S3200 with Fibre Channel
4
The Lenovo Storage S3200 can contain as many as 192 drives ranging from NL SAS to
SSDs depending on performance and storage capacity needs. The S3200 also offers
Intelligent Real-Time Tiering that can optimize storage performance, reduce cost, and
simplify management. Features also include SSD read cache, virtualized storage pools,
snapshots, and thin provisioning. With 6.4 GBps maximum sequential read throughput
and 5.3 GBps maximum sequential write throughput at a 512KB block size, the Lenovo
Storage S3200 can handle most workloads with a single dual-controller enclosure at an
affordable price.
Thin Provisioning
In supported operating systems (recent versions of Linux, VMWare, and Windows), thin
provisioning can create a very large volume that only uses the space on disk that is
required while leaving the other disk space for other volumes. This allows you to have a
setup that can expand and grow over time as needed and allows for several scenarios
that can save time and money.
Allows for provisioning a larger volume than should be needed to accommodate
any future growth that is unexpected
Fewer drives may be purchased initially with the plan for buying more as needed
o Drive prices reduce over time and the size of drives continues to
increase, meaning in 2 years you can purchase a larger drive for less
money
No more estimating what an application will need for storage
o A larger value can be used and if an application doesn't use it, no space
is wasted
We tested thin provisioning on the storage array for this validation.
Lenovo Storage S3200 with Fibre Channel
5
Single Copy on Write (SCW) Snapshots
The Lenovo Storage S3200 uses a snapshot technology called single copy on write
(SCW) with Original/Write Data Preservation capabilities. This technology reduces the
size of snapshots by preserving the data captured in a snapshot in place and then
writing to a new snapshot volume for all writes going forward. This significantly improves
speed of writes compared to methods such as copy on first write (COFW).
These snapshots allow rollbacks in the case of administrator error or a malicious user
overwriting information. Ransomware can be removed completely without a headache or
dealing with a criminal. It can also be very useful in a development environment to
create a point in time to roll back to after performing some actions to test a new
configuration and then to roll back in minutes to an old configuration.
We validated snapshots on the Lenovo Storage S3200 as part of this project.
Virtualized Storage Pools
Virtualized storage pools allow for flexible creation of storage volumes by writing
sections of data to different media types. When combined with Intelligent Real-time
Tiering, this allows data to be written to faster media when it is being accessed often and
to be stored on slower media when it is being accessed less frequently. In addition to
enabling performance gains, the virtualized storage pools simplify the management of
storage volumes. More storage can be added to the volumes via these storage pools
without requiring a full rebuild of a volume which saves time, money, and ensures data
integrity.
We used storage pools during this validation.
Lenovo Storage S3200 with Fibre Channel
6
SSD Read Cache and Tiering
Tiering occurs on the Lenovo Storage S3200 by separating drives into three types:
Archive – 7200 RPM drives
Standard – 10K or more RPM drives
Performances – SSDs (requires optional license)
Automatic tiering occurs among the different pools of drive types based on the frequency
of data access. Data blocks associated with frequently accessed files will migrate to the
highest performance tier available, while rarely accessed data blocks will be migrated to
a higher capacity tier. This process allows for users or applications to change their
access patterns and have the storage optimize itself every 5 seconds without any
administrator prevention.
The SSD read cache is another option for improving performance by expanding the
effective size of the controller cache without any additional licensing costs. Both
performance tiering and the SSD read cache were tested as part of this validation. The
tiering occurred at a granular level in 4MB chunks that allowed frequently accessed data
to be determined in a programmatic way without any user intervention. It is easy to
monitor where data is stored on the S3200 via either the command line or the GUI.
Lenovo Storage S3200 with Fibre Channel
7
Lenovo B300
The Lenovo B300 Fibre Channel Switch has up to 24 ports (8 ports licensed by default;
the remaining ports can be licensed in 8-port increments). The switch offers a rich set of
standard software features with the options to expand its capabilities as needed.
The B300 features the EZSwitchSetup wizard and can be configured in Access Gateway
Mode to simplify deployment. The switch provides full non-blocking performance with
Ports On Demand scalability to support SAN expansion and enable long-term
investment protection.
Lenovo B6505/B6510
The Lenovo B6505 Fibre Channel Switch can be configured as 12 or 24 ports while the
Lenovo B6510 Fibre Channel Switch can be configured as 24, 36, or 48 ports. Every
port in these configurations is non-blocking with a port-to-port latency of 700 ns while
running at 16 Gbps. This gives these specialized storage ports industry-leading, real
world benchmark numbers at scale. Beyond bandwidth and latency, these switches offer
manageability and ease of use through their multiple methods of setup including a CLI
and GUI with policy based alerting as an option to ensure that there are no outages for
critical data paths.
With support for Brocade's Fabric Vision™ technology both the B6505 and B6510 have
deep visibility into what is occurring on the storage fabric. An administrator can validate
physical components to ensure that no component is borderline or failing. This prevents
a cable from having a bend radius issue and degrading the performance of a whole
fabric over an important link. Additionally, beyond an initial validation, the administrator
can be alerted once policies have exceeded their limits and know about a problem
before users and their applications see the failures.
Lenovo Storage S3200 with Fibre Channel
8
With industry leading performance, simplified setup, and easy debugging and
troubleshooting, what more could be asked from a storage network?
Lenovo System x3550 M5
Leading the industry in reliability, Lenovo's most recent server offerings achieve
99.999% availability that equates to an average of 6.05 seconds of server downtime per
week. This reliability drastically reduces costs for both Lenovo and its customers,
allowing Lenovo to deliver cost efficient solutions at a lower price point.
The Lenovo System x3550 M5 is a 1U two-socket rack server that utilizes Intel Xeon E5-
2600 v4 processors, up to 1.5 TB of TruDDR4 Memory, and up to 12 drive bays to
deliver energy and space efficiency that matches or beats the best in the industry. The
server is designed for energy efficiency, including power supplies that deliver 80 PLUS
Titanium performance ratings. 80 PLUS Titanium is the highest achievable 80 PLUS
rating, which enables infrastructure, including breakers and UPS units, to be correctly
sized. They also save money over time by using less power.
The server also includes built-in reliability, such as Predictive Failure Analysis
and light path diagnostics to proactively monitor the server components and
identify failing parts. Lenovo servers ranked #1 in reliability for the third year in a
row, according to an independent ITIC 2015-2016 survey.
The Lenovo System x3550 M5 rack serverdelivers enterprise-class performance,
reliability, and efficiency at a competitive price.. .
Emulex 16Gb (Gen 6) Fibre Channel Host Bus Adapters (HBA) for System x
Lenovo Storage S3200 with Fibre Channel
9
Emulex 16Gbps (Gen 6) Fibre Channel HBAs deliver exceptional performance and
reliability for your Lenovo storage workloads. They seamlessly support Brocade
ClearLink™ diagnostics through Emulex OneCommand® Manager, ensuring the
reliability and management of storage network configurations when connected to
Brocade Gen 5 or Gen 6 Fibre Channel (FC) SAN fabrics. In addition, the adapters
ensure the highest I/O performance for mission-critical applications with Emulex
ExpressLane™ quality of service (QoS) that allow you to prioritize latency-sensitive or
flash storage traffic.
Lenovo Storage S3200 with Fibre Channel
10
The Emulex 16Gb (Gen 6) FC HBA features up to 1.6M OPS per port, the highest
available on a 16Gbps FC adapter. This is achieved through Emulex’s Dynamic Multi-
Core Architecture which dynamically scales HBA resources to any port that needs it,
essential when ports are used in active-standby mode. The Gen 6 HBAs can process up
to 71% faster SQL Server transaction and up to 50% faster data warehousing
transactions over 8GFC HBAs. They are ready for NVMe over Fibre Channel, which
accelerates network access to SSDs, supporting the upcoming NVMe over Fibre
Channel T-11 standard.
All Emulex adapters allow you to deploy faster and mange less with a unified
management tool (OneCommand® Manager) and common driver set.
Sizing Information
Server Sizing
Sizing a virtualized solution has numerous complexities for optimization that can be
taken into consideration for a completely optimal deployment. Numerous technologies
are on the market that can make this job significantly easier. VMWare® Distributed
Resources Scheduler (DRS) is the software product that enables VMWare to assist in
moving virtual machines to optimal locations in a cluster. With DRS, you can select to
allow migration to happen automatically or to merely monitor and suggest changes that
the administrator can allow or disallow. This feature is available in VMWare® Enterprise
and Enterprise Plus and simplifies both the initial sizing and understanding what
compute resources may be required going forward.
When a cluster is running DRS, server sizing requires a few simple yet complicated
questions:
What is the CPU capacity required for my applications?
How much RAM is needed for my applications?
Is there a requirement for a large amount of network or storage I/O?
If the current environment is not setup for virtualization yet, there are tools on both
Windows and Linux that can help with the sizing. Standard Linux tools such as sar, ps,
mpstat, top, and free can give a very strong idea on what may be required of a system's
virtualized equivalent when compared to the hardware that is inside a server. Windows®
has similar commands that can help with this transition such as PerfMon that will give
detailed information about how all systems are functioning. If already running VMWare®
vSphere on the cluster, esxtop is a good choice in gathering this information to help in
sizing.
Once this information has been gathered, it should be compiled together. Approximately
20% headroom for future growth or burst usage should be left as available. For RAM
Lenovo Storage S3200 with Fibre Channel
11
utilization, it is a simple matter of adding up memory usage across the servers and then
adding some overhead. For CPU utilization, it is most likely simplest to follow some
steps such as the below to determine what should be purchased:
Current CPU core GHz and quantity
Current utilization percent averaged across servers to be virtualized
Multiply current average utilization by quantity and GHz or current cores
o For ease of calculation, let’s assume we measured a 20% peak utilization
o Separate calculations for different server types or workloads may be done
and then added together
o 112 cores * 2.4GHz * .20 for utilization = 53.76GHz required
Multiply proposed CPU core GHz by core quantity in server for server total
o 3.0GHz * 20 cores (across 2 CPUs) = 60GHz
Multiply previous value on current setup by 1.20 for overhead and then divide by
servers
o 53.76GHz * 1.20 = 64.512GHz
o 64.512GHz / 60GHz per server = 2 servers required to safely run required
workload
To prepare for the event of a failure, an additional server should be available in an HA
cluster. It could be assumed that the second server in most scenarios with the above
calculation would be fine, so it is up to the discretion of the administrator familiar with the
workloads as to what his requirements are.
Network Sizing
Network sizing generally is very simple when utilizing a 10 Gbps network as it is unlikely
that that amount of bandwidth will be exceeded and even more unlikely when using the
commonly suggested method of bonding two adapter ports together to a form a 20 Gbps
trunk. If network usage is a concern, PerfMon on Windows and tools such as ntop or
cacti may be deployed on Linux.
Storage Sizing
There are two primary considerations when sizing storage in both a bare metal and a
virtualized environment. The most obvious consideration is storage capacity that has
several features related to it that can save a lot of money when purchasing drives. The
second consideration is the performance of the storage that can be the latency to reach
Lenovo Storage S3200 with Fibre Channel
12
that storage, IOPS, or throughput. Differences between IOPS and throughput can be a
consideration if the size of each transfer per IO is very small or very large. For this
exercise, we will assume 512KB blocks. Numbers will need to be adjusted accordingly if
your block size varies from this.
Storage Capacity Planning
When using thin provisioning as provided through the Lenovo Storage S3200 or
VMWare® VMDK thin provisioning, virtual storage pools should be used on the S3200.
Thin provisioning allows for new capacity on demand and has many feature
enhancements that make it the preferred option. With thin provisioning, storage usage
can grow over time as operating systems and the applications running on them grow
databases and logs.
For the initial purchase and requirements of a deployment, each guest hosted in a virtual
machine will be very different based on the sizing requirements of each operating
system and application. A few rules of thumb can help us know a good starting point and
can help us move forward to next steps until we can revisit specific numbers.
A typical full Windows® Server 2012 installation requires approximately 32
gigabytes for the operating system itself without deduplication, depending on the
version of Windows®
Each full Linux installation generally requires approximately 4 gigabytes for the
operating system itself without deduplication, depending on the distribution and
the options selected at install time (newer distributions may use 20GB or more)
Applications will give guidance on their specific storage requirements
The calculations are as simple as the OS install size combined with the application
install size requirements along with a small amount of overhead. For example, sizing a
web server running Nginx™ for storage would be done like the below:
Operating system install for CentOS® 7 requires 20GB
Nginx™ is included as part of the 20GB
Configurations or files to be executed and served will be a trivial amount extra
20% overhead would be 4GB
Provision more than 24GB for a simple Nginx™ web server or load balancing
setup on CentOS® 7
o Thin provisioning means we should expect 24GB to be used, but we can
create a larger volume
Lenovo Storage S3200 with Fibre Channel
13
Storage Performance Planning
Storage performance is an often overlooked aspect of planning an array. It is very
common either to provide too little performance for applications or to provide too much.
Applications will suffer with too little while budgets will suffer with too much. Performance
planning is simplified with hybrid arrays such as the Lenovo Storage S3200. Tools such
as iostat on Linux or PerfMon on Windows will allow you to view your applications
required IOPS. If using vSphere, you can use the performance charts to get a good idea
on what the storage performance requirements will be.
Sizing with an S3200 is made easier by having either an SSD read cache or
performance tiering to SSDs. These features allow the usage of high performance SSD
drives that will allow for thousands of IOPs per drive.
Lenovo B300, B6505 and B6510 considerations
The B300 is an entry level switch for small storage networks. For customers doing
deployments that use 18 or fewer servers and have no upstream SAN, the Lenovo
B6505 with its lower cost of acquisition, will be an optimal fit. If a customer has an
existing SAN, more than 18 servers, or plans on deploying multiple iterations (pods) of
the design detailed in this document, they should use a Lenovo B6510.
With the non-blocking nature of these Generation 5 (16Gbps) Fibre Channel switches, a
single switch configuration requires the consideration of how much bandwidth needs to
go to a compute node as well as the uplinks to storage. With 16 Gbps speeds, two links
to each controller on the S3200 should saturate the system and the same will occur on
the server. For multiple switch configurations, see the Brocade SAN Design and Best
Practices Guide in the Other Reading section.
Validation Setup and Diagram
The validation setup included three servers, two Fibre Channel switches, and one
storage array with a minimum configuration of drives. These drives consisted of four
nearline SAS drives and two SSDs for a performance tier. The performance tier license
was applied. Also included was 10 gigabit and gigabit Ethernet infrastructure for
management and node communication that is not included in this document. For
reference, a Brocade ICX 6430 and two VDX 6740 switches were utilized. For details on
what was tested, see the Bill of Materials section below. For a visual representation of
the storage configuration, see the figure below.
Lenovo Storage S3200 with Fibre Channel
14
Configuration Hints
Brocade
Three basic things must be done in any Fibre Channel configuration. There are
advanced steps including monitoring, optimization, and ensuring reporting of any errors
that are discussed in other documents. The three primary steps are:
Set an IP address
Update firmware
Configure zoning
Serial Port Setup
Lenovo Storage S3200 with Fibre Channel
15
The first step to configuring any of these settings is to connect via the serial console.
There is an included serial cable with the switch. This serial cable can be connected to
an adapter on a laptop or another server. A popular serial terminal program is PuTTY,
which can be downloaded from the web, and there is a link in other reading. Screen is a
favorite Linux serial emulator of many administrators and can be used. The serial
settings in PuTTY only require changing the COM port to match what your adapter is set
to. To learn what COM port, navigate to the Device Manager that can be accessed via
running the command mmc devmgmt.msc or navigating through standard menus.
Logging In, Setting the IP Address and Updating Firmware
Once you have accessed the switch's serial console, you can login with the default
password for the account. If you are planning to script any zoning, log in as root with the
password “fibranne” by default. If you will be doing manual zoning or zoning through the
WebTools interface, you can login as admin with the password “password” by default.
These passwords should be changed, which can be done on the command line with the
passwd command. The syntax is simply passwd admin and then following prompts.
Lenovo Storage S3200 with Fibre Channel
16
Both the commands we need for the next steps can be menu driven. There is a
verification step in between we can run to ensure that our settings are correct. The
commands we want to run are in the list below.
Ipaddrset
ping $IPADDRESS
firmwaredownload
o As a note, firmwaredownload is optional and for most environments may
not be required
Answer the menu driven prompts and you will have a switch that is ready to be
managed.
Zoning
The below are command examples for setting up a simple zoning configuration:
alicreate "Example_Alias_1", "xx:xx:xx:xx:xx:xx:xx:xx"
zonecreate "zone_Example_1", "Example_Alias_1; Stor_Example_Alias_1;
Stor_Example_Alias_2"
cfgcreate "lenovo","zone_Example_1; zone_Example_2; zone_Example_3"
cfgsave
cfgenable "lenovo”
Lenovo Storage S3200
By default, the S3200 has a static IP address configured which will be active unless
there is a DHCP assignment given to it. The default IP addresses are 10.0.0.2 and
10.0.0.3 with a /24 CIDR (255.255.255.0 netmask.) In an environment that has DHCP,
you can learn the IP assignment by checking the system logs or you can simply directly
attach an Ethernet cable to a server or laptop with an address assigned to the interface
such as 10.0.0.4 with a 255.255.255.0 netmask.
Once you can establish connectivity, there are two primary options of setting up the
storage. The option displayed in this guide is the web interface, but the CLI can achieve
the same result. See the Other Reading section for additional documentation including
the CLI reference guide.
Lenovo Storage S3200 with Fibre Channel
17
The first page to greet you will appear as the below. Note that approved optics must be
used in the ports. Otherwise, they will be disabled as is shown in the image.
The first step in configuring your array is to set up disk groups. You can divide these
groups up later to individual nodes via the creation of volumes and you should combine
drives into pools. There are performance considerations for assigning a large number of
drives into a single disk group and having every node in a single pool where the IO
blender (multiple servers at once accessing the same resource) can lead to significant
random reads and writes. Because of this, size the pools appropriately and keep in mind
additional disk groups can be added in the future. It also should be noted that having
pools on the separate controllers would ensure load balancing. Because of this, having
at least an A and a B pool is considered best practice.
Lenovo Storage S3200 with Fibre Channel
18
Additional disk groups can be added to a single pool. In the case of performance tiering,
ensure that it is added to the same pool as the other disk groups.
Once your disk groups are created and assigned to their pools, volumes should be
created. You can create multiple copies of a single volume sizing and select which pool
to draw from. In our case, we created three to be used on our three servers.
Lenovo Storage S3200 with Fibre Channel
19
The final step to a basic storage setup is to map the volumes to the hosts. Select the IDs
that are sequential and map them to the correct volumes. You can verify that the WWNs
displayed in the ID field are unique by checking on the host.
Lenovo Storage S3200 with Fibre Channel
20
Bill of Materials
Table 1 – What was tested and what Lenovo recommends for a basic configuration involving the
S3200 and Fibre Channel
Part Number
Description
3873AR4
Lenovo B6505 FC SAN Switch by Brocade
88Y6393
Brocade 16Gb SFP+ transceiver module
39Y7932
4.3m, 10A/100-250V, C13 to IEC 320-C14 Rack Power Cable
00MN502
Lenovo 1m LC-LC OM3 MMF Cable
00MN505
Lenovo 3m LC-LC OM3 MMF Cable
5463EFU
TopSeller x3550 M5, 2x Xeon 10C E5-2650v4 105W
2.3GHz/2400MHz/25MB, 4x16GB, O/Bay HS 2.5in
SATA/SAS, SR M5210, Multiburner, 2x550W p/s, Rack
00AJ136
500GB 7.2K 6Gbps NL SATA 2.5in G3HS HDD
01CV840
Emulex 16Gb (Gen 6) FC Dual-port HBA
00KA066
System x3550 M5 PCIe Riser 2, 1-2 CPU (LP x16 CPU0 + LP
x16 CPU1)
00JY820
Emulex VFA5 2x10 GbE SFP+ PCIe Adapter
64116B2
TopSeller Lenovo Storage S3200 LFF Chassis Dual FC/iSCSI
Controller
00WE790
Lenovo Storage 3.5in 8TB 7.2k NL-SAS HDD
00WC030
Lenovo Storage 3.5in 400GB SSD SAS (2.5inin 3.5in)
00WC089
Lenovo Storage S2200/S3200 16G Fibre Channel SFP+
Module 1 pack
00WE764
Lenovo Storage SSD Data Tiering Upgrade
Cabinets and Power
Lenovo sells multiple cabinets and power distribution units (PDUs) for deployments that
include large scale data center designs to setups that reside at the desk side or closet in
a small office setting. For a full listing of cabinet and power options, refer to the Other
Reading section Server Cabinets for System x and Power Distribution Units for System x
Servers.
Conclusion
Lenovo has collaborated with leaders in the industry to create architectures and
solutions for customers who need a solution for their datacenters to their server closets.
The Lenovo Storage S3200 has been validated with Brocade and Emulex technology to
prove that everything functions optimally. This reduces risk in deployment, increases
velocity for project deployment, and helps to ensure that capital spending is at a
minimum with correct sizing. The Lenovo Storage S3200 with Fibre Channel provides a
simple step up when NAS arrays are pushed to their limits and scales up to the
Page is loading ...
Page is loading ...
/