Numeral systems, measurement units and capacity

On September 23^rd, 1999,  NASA lost contact with its Mars Climate Orbiter (MCO) as it burned up unexpectedly on the day that should have ended up in celebration of it entering Mar’s orbit. The failure was due to one team using English units (e.g. inches, feet, and pounds) while the other used metric units (e.g. centimeters, meters, and kilograms) for key spacecraft operations that steered the MCO through space. Instead of putting the MCO into Mars’ orbit, the failure put it on a trajectory too close to the planet with the result being it burned up in the Martian atmosphere.

This may be one of the most radical examples of what can happen when organizations do not clarify which measurement units are being used but it serves the purpose of highlighting the importance of understanding this critical factor. To draw a parallel with IT, organizations that don’t understand which measurement units have been expressed could end up with too much or too little capacity for data in transit, storage or both.

Numeral and measurement systems have had internationally defined standards as far back as the 19^th century. Organizations such as the Internal Standards Organization (ISO) and the International Bureau des Poids et Mesures (BIPM), (known in English as the International Bureau of Weights and Measures) maintain and update these in conjunction with many other global organizations. Other organizations such as the International Electrotechnical Commission (IEC) and the Institute of Electrical and Electronics Engineers (IEEE) have worked with the ISO and the BIPM to define specifics for the field of IT.

The standards used in IT for decimal multiples are called the International System of Unit (SI Unit) prefixes and at time of writing (December 4^th, 2017) are documented in the 8^th edition, 2006  of the International System of Units brochure available at the following link: https://www.bipm.org/utils/common/pdf/si_brochure_8.pdf#page=127
This edition recognizes that the SI Unit prefixes should not be used when expressing binary multiples. Instead, the adoption of prefixes for binary multiples as defined in IEC 60027-2 and the IEEE should be used in the field of IT to avoid the incorrect usage of the SI prefixes. IEC60027-2 has since been superseded by IEC 80000-13:2008 IEC - SI Zone > Prefixes for binary multiples

Tables 1 & 2 detail the SI Unit (Decimal) Prefixes and IEC (Binary) Prefixes, including their names, factors, and origins.

Value (No. bytes)	Unit Prefix		Name	Factor	Origin
	Name	Symbol
1,000	kilo	kB	Kilobyte	103	thousand
1,000,000	mega	MB	Megabyte	106	million
1,000,000,000	giga	GB	Gigabyte	109	billion
1,000,000,000,000	tera	TB	Terabyte	1012	trillion
1,000,000,000,000,000	peta	PB	Petabyte	1015	quadrillion
1,000,000,000,000,000,000	exa	XB	Exabyte	1018	quintillion

Table 1 SI Unit Prefixes (Decimal)

Value (No. bytes)	Unit Prefix		Name	Factor	Origin
	Name	Symbol
1,024	kibi	kiB	Kibibyte	210	Kilobinary ~thousand
1,048,576	mebi	MiB	Mebibyte	220	Megabinary ~million
1,073,741,824	gibi	GiB	Gibibyte	230	Gigabinary ~billion
1,099,511,627,776	tebi	TiB	Tebibyte	240	Terabinary ~trillion
1,125,899,906,842,620	pebi	PiB	Pebibyte	250	Petabinary ~quadrillion
1,152,921,504,606,850,000	exbi	XiB	Exbibyte	260	Exabinary ~quintillion

Table 2 IEC Prefixes (Binary)


Using the data from Table 1 and Table 2 let’s say we express our need for a new capacity requirement as 512 TB, as opposed to the actual capacity requirement of 512 TiB.512TB, equals 512,000,000,000,000 bytes, whereas 512TiB equals 562,949,953,421,312 bytes.

Just like the error that led to the failure of the MCO, in this example using the incorrect measurement unit would result in a shortfall of 50,949,953,421,312 bytes, or 46.34TiB.
Now, there is some good news for the capacity ranges that are common on today’s storage platforms that may not result in something as catastrophic as the Mars example!  Typical primary storage requirements can be measured in or around trillions of bytes (TB or TiB) with a growing number creeping into the quadrillions (PB or PiB).

At these ranges, the margin of error if the measurements are mistaken is that you’ll either lose or gain up to 9.95%.  On its own, this may not necessarily represent a problem as some (but not all) solutions will observe just marginally more than what has been requested to cover things like pool reserved capacities or drive protection overheads.  However, if you’re on the side of the 9.95% shortfall and this gets compounded with an assumption on data reduction technologies that is beyond what the system can do for the workload in question – you may find the solution short on capacity right out of the gates.

Resource Management Features of the Symmetrix VMAX for Cloud Providers

Pre-2018 post from old blog...please check support.emc.com for latest information.

As cloud computing continues to consolidate more and more multiple, disparate workloads into our compute, network and storage platforms there is now an ever growing need to control resource management at various different component layers within each stack.

Over the years the EMC has developed a portfolio of powerful, trusted features for the Symmetrix that enables cloud providers to dynamically manage resources both inside and outside the array whilst delivering a compelling total cost of ownership (TCO) that is required to compete in this utilities market.

• At the front end of the VMAX we have Auto Provisioning Groups and Host I/O Limits. Auto Provisioning Groups facilitate rapid change management for storage presentation and reclamation activities aka LUN masking through the use of groups of HBAs, targets and LUNs. Here providers have the ability to dynamically manage, scale and balance configurations for their subscribers. Host I/O Limits are a feature of Auto Provisioning Groups that provide the ability to place workload limits on particular services or subscribers.
• In the mid section of the array we have a powerful feature called Dynamic Cache Partitioning (DCP) that enables logical separation of cache.
• At the back end we have Fully Automated Storage Tiering for Virtual Pools (FAST VP) which is a truly awesome feature that leverages the difference in IO densities and capacities of flash and traditional spinning rust to provide a better response time at a lower cost by moving your most active data to the most appropriate performance tier and your inactive data to a bulk storage tier! Of all the features the Symmetrix has this is by far my favorite as it delivers so much operational efficiency coupled with numerous environmental and and cost benefits. For cloud or multi-tenant providers that have subscribers with varied workloads this is a must have.
• Federated Tiered Storage (FTS) enables providers to add an external storage tier from any supported EMC or 3rd party array that can avail of all the mature Symmetrix features like SRDF, TimeFinder etc.

Symmetrix VMAX Auto Provisioning Groups

Pre-2018 post from old blog...please check support.emc.com for latest information.

For those new to Symmetrix VMAX, Auto Provisioning is probably what you'll assume has been the norm for Symmetrix administrators for years but if you've experienced provisioning on the previous models with many devices to large clusters over many front end ports you'll recall either the endless hours you put in to ensure everything was mapped and masked properly.

Those days are gone!

With Auto Provisioning the entire provisioning mechanism has been vastly simplified and is now extremely rapid with some new SYMCLI commands and presentation wizards in Unisphere for VMAX.

Auto Provisioning Groups are a very welcomed paradigm shift for storage administrators during the presentation and reclamation of devices from the array and there are a few concepts to first understand and we'll cover each of them in turn before getting into Auto-Provisioning itself.

If you've been doing this on Symmetrix prior to VMAX then skip the first two sections as they will only tell you how to suck eggs!

Symmetrix Device Mapping and Masking 101

For a Symmetrix Logical Volume which I'll call a symvolid to be presented to a host it must first be mapped to a front-end port and given a channel address which we'll call a LUN number - this task is known as mapping.

Once the devices have been mapped up to a front-end port and given a LUN number an HBA must then be granted access to it - this task is known as masking and on DMX the front-end ports must be VCM-enabled and on VMAX ACLX-enabled. When a port is VCM or ACLX-enabled this means device masking must be carried out to grant access.

On Symmetrix it is recommended to map the symvolid to at least two front-end ports and then mask it to at least two HBAs.  Prior to Auto-Provisioning this procedure took a few commands each of which took up to 20 minutes to perform on the array.

With Auto-Provisioning - storage can be presented to even the largest of clusters in minutes.

Channel Addressing and Dynamic LUN Addressing

With Solutions Enabler 7.0 and Enginuity 5772 the Symmetrix gained a feature known as Dynamic LUN Addressing (DLA) which enables storage administrators to dynamically assign LUN addresses on a per HBA level - and now at the initiator group level on VMAX.  Previously you could only ever use the channel address assigned to the device on the front-end ports and this at times caused havoc for those operating systems like Windows and ESXi that are restricted to an address range of 00-FF (0-256).  If you had a Windows box and masked out a device with a LUN number higher than FF then you simply couldn't discover the device as the address range is beyond what Windows can natively address. DLA is optional on 5772 and 5773 but is default on Auto-Provisioning (5874 and higher) and empowers storage administrators with the ability to serve hosts with restricted address ranges on FA ports that have more than 256 devices. In a nutshell, with DLA it really doesn't matter what channel address is assigned to the device on the port as DLA will modify this at an initiator group level.


...enter Auto-Provisioning Groups

Auto-Provisioning introduces device management by joining groups of objects in a masking view. Lets take a look at the groups. In the examples shown we'll have a two node ESXi cluster - each host with its own unique boot from SAN device and a shared device accessible by both hosts. Host 1 will have four local devices and Host 2 one.


Initiator Groups contain the World Wide Port Name (WWPN) objects of the HBAs used by your host or cluster.

Auto-Provisioning: Initiator Group

Auto-Provisioning: Cascaded Initiator Group

• a VMAX can contain up to 8192 initiator groups
• cascaded initiator groups allow for groupings of initiator groups and can only go one deep - a real handy feature for clusters
• cascaded initiator groups have a maximum of 1024 initiators total
• a maximum of 32 Fibre Channel initiators or 8 iSCSI names per group or a combination of both per group
• an initiator can be a direct member of only one group
• an initiator group can be a member of more than one masking view
• port flags can be set at the initiator group level
• in general its more efficient to group all HBAs from one host in an initiator group
• the consistent_lun parameter can be used to ensure the same address is assigned to a device presented to an initiator group and is generally what I use
• akin to the Hostss folder on Navi/Unisphere Storage Groups
• suggested naming conventions: IG_HOSTNAME or CIG_CLUSTERNAME

We'll first create the initiator groups unique to each ESXi host for the local devices and another cascaded initiator group to use for the shared devices masking view that will be presented the cluster shared storage to both hosts.

symaccess -sid 1234 create -type initiator -name IG_ESXiNODE1 -consistent_lun <<< create the group and name it for the host
symaccess -sid 1234 -type initiator -name IG_ESXiNODE1 add -wwn 211100243581230d <<< add the WWN of the first HBA to the group for the host
symaccess -sid 1234 -wwn 211100243581230d set hba_flags on D -enable <<< toggle any port flags at the initiator level
symaccess -sid 1234 -type initiator -name IG_ESXiNODE1 add -wwn 211100243541230e
symaccess -sid 1234 -wwn 211100243541230e set hba_flags on D -enable
symaccess -sid 1234 create -type initiator -name IG_ESXiNODE2 -consistent_lun
symaccess -sid 1234 -type initiator -name IG_ESXiNODE2 add -wwn 211100243581123f
symaccess -sid 1234 -wwn 211100243581123f set hba_flags on D -enable
symaccess -sid 1234 -type initiator -name IG_ESXiNODE2 add -wwn 211100243541123e
symaccess -sid 1234 -wwn 211100243541123e set hba_flags on D -enable
symaccess -sid 1234 create -type initiator -name IG_ESXiCLUS -consistent_lun <<< create the cascaded initiator group and name (this is the parent object)
symaccess -sid 1234 -type initiator -name IG_ESXiCLUS add -ig IG_ESXiNODE1 <<< add the initiator groups (these are the child objects)
symaccess -sid 1234 -type initiator -name CIG_ESXiCLUS add -ig IG_ESXiNODE2


Port Groups contain front-end port objects from the array that are to be used by your host or cluster.

Auto-Provisioning: Port Group

There are generally two ways to implement port groups and I generally term these as rigid or elastic. Rigid being where you configure a fixed number of port groups with a fixed number of ports. Elastic being were you have a defined or modeled port group layout but create a new port group for every host based on these definitions or models and can grow or shrink these on a per host basis by adding or removing ports. Each have pro and cons from a day to day management perspective but my preference would be to go with the elastic approach.

• a VMAX can contain up to 8192 port groups
• the maximum number of ports equals maximum number of ports in the VMAX - currently 128 for an 8-engine box
• ports can be a member of more than one group which is handy if you decide to implement a port group per server or cluster
• port groups can be a member of more than one masking view
• for availability and performance port groups should contain a minimum of two ports from different directors and with thin provisioning a minimum of four ports should be considered - remember that for platforms like ESXi increasing the number of ports beyond four decreases the maximum number of LUNs the host can address as it 256 LUNs / 1024 paths.
• loosely akin to the advanced HBA registration in Navi/Unisphere Storage Groups
• suggested naming conventions: PG_NAME (hard) or PG_HOSTNAME (soft) or PG_CLUSTERNAME (soft)

In this example we will create three hard port groups on this VMAX - two we'll use for the local device masking (each with four ports) and the third (with eight ports) for the cluster shared device masking.

symaccess -sid 1234 create -type port -name PG_01_4MEM <<< create the 4 member port group
symaccess -sid 1234 -type port -name PG_01_4MEM add -dirport 8E:0 <<< add ports to the group
symaccess -sid 1234 -type port -name PG_01_4MEM add -dirport 7E:0
symaccess -sid 1234 -type port -name PG_01_4MEM add -dirport 9E:1
symaccess -sid 1234 -type port -name PG_01_4MEM add -dirport 10E:1
symaccess -sid 1234 create -type port -name PG_02_4MEM
symaccess -sid 1234 -type port -name PG_02_4MEM add -dirport 8F:0
symaccess -sid 1234 -type port -name PG_02_4MEM add -dirport 7F:0
symaccess -sid 1234 -type port -name PG_02_4MEM add -dirport 9F:1
symaccess -sid 1234 -type port -name PG_02_4MEM add -dirport 10F:1
symaccess -sid 1234 create -type port -name PG_07_8MEM <<< create the 8 member port group
symaccess -sid 1234 -type port -name PG_07_8MEM add -dirport 8E:0 <<< add ports to the group
symaccess -sid 1234 -type port -name PG_07_8MEM add -dirport 7E:0
symaccess -sid 1234 -type port -name PG_07_8MEM add -dirport 9E:1
symaccess -sid 1234 -type port -name PG_07_8MEM add -dirport 10E:1
symaccess -sid 1234 -type port -name PG_07_8MEM add -dirport 8F:0
symaccess -sid 1234 -type port -name PG_07_8MEM add -dirport 7F:0
symaccess -sid 1234 -type port -name PG_07_8MEM add -dirport 9F:1
symaccess -sid 1234 -type port -name PG_07_8MEM add -dirport 10F:1


Storage Groups contain device objects from the array that are to be presented to your host or cluster.

Auto-Provisioning: Storage Group

• a VMAX can contain up to 8192 storage groups
• the maximum number of devices per group is 4096
• a device can be a member of more than one storage group
• a storage group can be a member of more than one masking view
• devices are assigned dynamic LUN addresses using the Dynamic LUN Adressing (DLA) feature
• used by FAST DG and FAST VP to associate the content of a storage group with a tiering policy
• akin to the LUNs folder on Navi/Unisphere Storage Groups
• suggested naming conventions: SG_HOSTNAME or SG_CLUSTERNAME

In this example we have three storage groups: two for the unique local devices and one for the cluster shared devices.

symaccess -sid 1234 create -type storage -name SG_ESXiNODE1 <<< create the storage group for each host
symaccess -sid 1234 create -type storage -name SG_ESXiNODE2
symaccess -sid 1234 -type storage -name SG_ESXiNODE1 add dev 11A <<< add devices to the storage group
symaccess -sid 1234 -type storage -name SG_ESXiNODE1 add dev 11B
symaccess -sid 1234 -type storage -name SG_ESXiNODE1 add dev 11C
symaccess -sid 1234 -type storage -name SG_ESXiNODE1 add dev 11D
symaccess -sid 1234 -type storage -name SG_ESXiNODE2 add dev 222
symaccess -sid 1234 create -type storage -name SG_ESXiCLUS <<< create the storage group for the cluster shared devices
symaccess -sid 1234 -type storage -name SG_ESXiCLUS add dev 333 <<< add devices to the cluster shared devices storage group


Masking Views knit these groups of objects together on the array and automatically perform the mapping of the devices in the storage group to the front-end ports in the port group and the masking of the devices in the device group to the initiators in the initiator group.

Note this example shows example names used in the SYMCLI examples.

Auto-Provisioning: Masking View

• a single view can only contain one initiator group, one port group and one storage group
• akin to Navi/Unisphere Storage Groups
• suggested naming conventions: MV_HOSTNAME or MV_CLUSTERNAME

The final SYMCLI command we need to run is to create the view. Here we have three views: one for the local storage on each node and one for the cluster shared devices.

symaccess -sid 1234 create view -name MV_ESXiNODE1 -sg SG_ESXiNODE1 -ig IG_ESXiNODE1 -pg PG_01_4MEM <<< create the masking view for the host
symaccess -sid 1234 create view -name MV_ESXiNODE2 -sg SG_ESXiNODE2 -ig IG_ESXiNODE2 -pg PG_02_4MEM
symaccess -sid 1234 create view -name MV_ESXiCLUS -sg SG_ESXiCLUS -ig CIG_ESXiCLUS -pg PG_01_8MEM <<< create the masking view for the cluster shared devices

New Storage Presentation and Reclamation

So when you've configured your masking view and your cluster is now in production you get a request to present more storage. With Auto Provisioning to present additional devices all you have to do is add the device into the storage group and Auto Provisioning will automatically map and mask the device for you.

symaccess -sid 1234 -type storage -name SG_ESXiCLUS add dev 786  <<< new device presented to all cluster nodes in one command

Storage Reclamation is also fairly straightforward. If you need to remove a device or a number of devices from a host or cluster you simply remove the device from the storage group and you have the option to unmap in the same command.

symaccess -sid 1234 -type storage -name SG_ESXiCLUS remove dev 786 -unmap  <<< device removed from all cluster nodes and unmapped from all front-end end ports


Further information on Auto Provisioning and Dynamic LUN Addressing can be found @ http://support.emc.com/

Symmetrix Virtual Provisioning (VP) Migrations and Space Reclamation

Pre-2018 post from old blog...please check support.emc.com for latest information.

So I've put on a bit of weight over the last few years and when I talk about thick to thin in a storage context I wish I could apply a thin migration plan to myself but alas the only solution for that is exercise which I don't do enough of at the moment!

Anyhow - migrating to thin pools is something that comes up in discussion on a regular basis and over the last few months I've been privileged enough to author migration plans for and use in anger most of the great tools that EMC have to ensure customers get the very best out of that minuscule 768KB sub-extent size utilized by the Symmetrix VMAX!

Typical migrations usually consist of loads of traditionally provisioned storage arrays that are being consolidated into a new EMC Symmetrix VMAX which in most environments these days is being configured with VP pools were customers are leveraging other technologies such as Virtual-LUN and FAST VP or just simply VP itself.

For those who intend to or are leveraging virtual provisioning pool over-subscription the chances are that you're going to want to strip out the unused zeroes during any inbound data migration......and EMC provide a range of array and host level tools to facilitate this objective.

Array-based migrations to thin pools

On the VMAX itself there are currently two migration methods that you can use to strip out those unwanted zeroes and the products that facilitate this are SRDF and Open Replicator. For those of you who have been around Symms these products have been available for years and are probably the most tried and tested replication products in the enterprise storage market and in VMAX the developers have added the ability to reclaim zeroed space in flight with a few caveats. With SRDF, you must currently use adaptive copy mode and with Open Replicator the control volumes must exist on the Symmetrix VMAX and these methods are the perfect fit for migration purposes.

Host-based Migrations to thin pools

If your migration plan can't leverage an array based method then you're going to ask the host to move the data, which is not ideal as the array should do this for you but is sometimes a necessity for one reason or another. EMC have the tools in this space to help and these are currently Open Migrator/LM and PowerPath Migration Enabler (Host Copy).

Open Migrator/LM is a very powerful product that in its simplest form copies an existing volume to a new VMAX volume and once in sync you choose a time to swap the target device over to be the new source - simple as that. If you are migrating Windows Server 2003 or 2008 machines to your new VMAX and need to correct partition alignment and/or NTFS cluster size then this product can do both - with the applications online.

PowerPath Migration Enabler (Host Copy) is a great feature of PowerPath. Assuming you've PowerPath patched to supported levels you can leverage PowerPath to facilitate an online migration for any device managed by PowerPath irrespective of whether its a LUN from an HP, IBM, EMC or any supported array.

It's also worth noting that many third party vendors at this point including VMware and Symantec support thin reclamation initiatives and recent versions of Storage Foundations and ESXi support various thin-aware features like SmartMove and Storage vMotion that can remove unsed zeros during migrations.

Conclusion

Some of the benefits behind reclaiming contiguous zeroes in flight during a migration are that it minimizes capacity consumption and reduces the load on the array as only extents with data are actually being written to disk. All in all these migration methods can be used to reduce or eliminate the downtime usually associated with such migration activities and reclaim valuable unused capacity.

Those who can't take advantage of these thin friendly migration tools to Symmetrix VMAX you do have the option to run an online Zero Space Reclaim (ZSR) post traditional migration methods which releases blocks of contiguous unused extents back into the pool for reuse.