Given “enough” money, what would you have if you can dream to build data center?
What type racks?
What type of fire suppression system
What type of physical security system
What type data security system
What kind of power backup
What type of routers, switches, etc
What type pipe feeding your net connection
? 977 503? 977 503?
Best Answer: 64-bit Windows 7
8 gigs of DDR3 RAM
GeForce GTX 250
500 gig SATA HD
Some type of Gigabyte or Asus BTX motherboard
Two DVD-RW disk drives
Oh wait, limited budget…….
Re:Like I said, dream systems, as if it was the only correct way to do things.
Please keep discussing this.
Re:I think Rittal makes the cabinets for APC. Whether they do or don't (for APC) – Rittal makes *THE BEST* cabinets and racking systems.
If the system is going into a disaster-prne area (i.e., somewhere on the fault line), then the Generator should be running on diesel fuel. A 400KW Cat generator will run ~12 hours on 200 gallons (the self-contained fuel tank) under full load. Diesel is preferred over LP and Nat gas (Nat Gas can have the line severed in an earthquake, LP gas lines are suseptable to shearing and they're under pressure – Diesel lines are not under pressure and can be reinforced / engineered to survive stretching and movement). Generally, if you have a generator, you have it serviced by an outside contract, and have the fuel deleivered on a regular basis while it's in operation. A diesel gen can also run on kerosene or cooking oil if for some reason the deliveries cannot be made).
Serious data centers will have Level 5 diversity (I believe that's what it's called) for the network trunks: meaning that there are (at least) two seperate trunks, coming from two separate paths (as far apart as possible) entering the building from two different points (i.e., North wall, South Wall). In addition, the service contract should include verification from the provider(s) that the trunks DO NOT ever join in a single path anywhere along the routes. The startup and carryover time is covered by an online UPS system (needs to carry the full projected load for (at least) 8 seconds, generally spec'd for much longer).
An OC48 (2.4G) or OC192 (~10Gig) would be a minimum pipe. Maybe an OC12 (622Mbps) for a small center.
Physical security is a page or two all by itself. The super-condensed version: Reinforced concrete block exterior, no windows, man-trap entrances / exits, manned & moitored 7X24. The trap space can have systems to measure weight and/or scent, and should include a metal detector. The outgoing measurements are compared to the incoming measurements, and both are compared to the information kept on the employee. Passage begins with a biometric badge system, verified by the monitor / guard for that passage. The security center is a separate cell within the building, or off-site, or both.
Routers / Switches : Who cares…. boxes is boxes 
.
(Actually, I agree, Cisco, Foundry, Extreme …)
There's a jillion other little things – type of concrete, reenforcement systems, power distribution, types of doors, water filtration systems, air / environmental systems, food storage, emergency areas (first aid centers / hospitals, etc) …. the point being that there's Data Centers and then there's "hardened" Data Centers – meant to be self-sufficient in case of disaster, flood, riot, terrorist attack, etc. If you want to go the whole nine yards, you can.
Depending on the specific role of the facility, there might be some changes to the bill-of-materials.
JM.02
Scott
Re:Yeah, make facilities do all the hard stuff but ensure they follow your standards and design…
power, physical security, etc.
Power being the most important. At least 2 power distribution units to each rack (why have redundant power supplies if they are attached to the same PDU/circuit).
Run DC for all comms gear, 240v AC for all host gear.
Re:Here's my go at it. I've designed and managed the build-up of two data centers for a telephone company. There's a few things to consider.
First, the physical facilities themselves (the room, etc.) need to be totally separate from the network. A good data center will last decades. The network infrastructure that you put in there will last 4 years, if you're lucky. Designing the network is fairly easy compared to designing the data center itself. So, I'll answer on the data center.
What I've done..
What type of racks?
Totally depends on what you're putting in there. A lot of people like enclosed cabinets (looks neater) and some like open racks. I prefer a mx of the two – Cabinets for servers, racks for network gear and patch panels. I really like the APC cabinets (OEM to Dell, Compaq, etc.) for PC gear and the Shark racks for Unix boxes – Good ventilation.
What type of fire suppression system
Two things here: FM200 for chemical retardant (first line of defense). Your second (and absolute last) is a dry-pipe sprinker system. Dry pipes are crucial – If you happen to swing a ladder and take out a sprinkler head, it won't set off the whole system and flood your data center.
What type of physical security system
At my current job, we use a a two-tier security system – Your typical cardswipe/HID card entry system at the door and then a second mesh "cage" around the most critical network components that only certain people have access to. IE, Joe server tech can't go plugging his box into the backbone switch on his own.
What type of data security system
Eeeh. That's part of the network design, not part of the data center.
What type of power backup
Each rack should have at LEAST two (usually 4+) circuits, each run off of a separate PDU (Power Distribution Unit, think big breaker box). PDU's are connected to UPSes, which are connected to a diesel generator with plenty of fuel. If at all possible, you should have redundant UPSes and generators. Speaking of generators, they aren't any good without fuel. Get a contract with a reputable supplier that can deliver fuel to you in the time you need to survive. Always keep 24 hours of fuel on hand. Your supplier should be able to deliver within 18 hours, then every 18-24 hours thereafter. If at all possible, keep an 8 hour margin of fuel to spare, if necessary. Just remember that you are a lower priority compared to hospitals, telcos, etc. and might not get fuel, if things go bad.
A central shutdown system is key, too. You should have something that sends a signal to your servers to gracefully shut down when the UPS is down to ~15 minutes of run time. (You should have ~30 minutes of normal runtime and generators start up and take over in about 10 seconds. If you get to 15 mins, the generators are dead and things are going down hard, so shut down).
What type of routers, switches, etc
Again, network trivialness.
What type of pipe feeding your net connection
Again, network design issues. Only thing to consider here is that if it's mission critical, you should have multiple entrances into the building on the SONET ring (We had an OC48 ring, lit with OC12's on three buildings and the CO's).
Building two data centers in two years was pretty much a full time job for three people – One techie, one facilities guy and one PM. That doesn't count the crew of 10+ people doing the work. It's a massive undertaking.
I have some pics from my "little" data center (5,000 SF) if you're interested. Nothing too detailed, just show shots of the floor.
- G
Re:Luxemburg data center? pics? links?
Re:Go look at Worldcom's Luxemburg data center. Do that.
Re:Well, I work for a major manufacturer/retailer, and here's how our datacenter works (I'd say it's near perfect, and it definitely cost A LOT of money – I'd guess easily in the tens to hundreds of millions of dollars):
1. Racks – not sure
2. Fire suppression system – datacenter has raised floor with some sort of gas that sprays out under the floor to put out the fire. Not sure if it's fatal or not hehe
3. Physical security – Only authorized users can get in and must swipe a card. Cameras throughout the building and datacenter are always recording any activity.
4. Data security – data is backed up constantly to 2 large tape silos on another floor of the building, in addition to another location in a different city. The tape silos have robotic arms that fly around and retrieve tapes on demand.
5. Power backup – not sure exactly what kind of system we use, but I know we have enough backup power to run for a week or so. I believe a lot of the backup power comes from very large UPSs (think room-sized batteries).
6. Routers/switches – we use all Cisco routers and switches. We run a completely redundant network (basically like running 2 networks in parallel). If a switch or router goes out somewhere, there is a backup that is working immediately. All the switches/routers are connected by gigabit ethernet or fiber.
7. Net connection – we have connections to 3 of the largest carriers, and they all use a separate backbone connection to the net. So if one goes out, we have 2 to rely on still.
As far as servers go, all servers have redundant power supplies, RAID, etc. so if any part of a server breaks, it will usually stay up and running. With all the high-end equipment and our fully redundant *everything*, we have been able to boast network/server uptime that is better than a majority of the best ISPs. Of course, we also have 150 people working in our IT department to support everything
Re:What if you had to direct the facilities as to what to do?
Then I wouldn't work for IT, I'd work for Facilities/Physical Plant.
In which case I'd probably do whatever the unions told me 
bart
Re:What if you had to direct the facilities as to what to do?
Re:It depends.
I built many very nice datacenters and they all follow a standard rule…
"make it managible and make it scale"
racks – who cares
fire supression system – who cares, that is for facilities to do
physical security – facilities again with ITs requirements
data security – not sure what you mean
power – generator with 7 days fuel
routers, switches – cisco, foundry, extreme
type of pipe – three big ones to AT&T, L3, worldcom
0 Comments.