Building your own lab
-
While looking for some stuff on ebay I realized that you could build your own multi-server lab for very little money - if you know what to look for.
$1300 will give you 4 independent servers in 2U of space.
A total of 64 cores @ 2.6Ghz and 256GB RAM.- 4 node 2U Supermicro server $500
- Eight 8 core Xeon E5-2650v2 CPUs $400
- 16x16 GB=256GB PC3-12800R RAM $400
PS. You can run it with a single CPU per node as well. Each node will then have 8 cores and 64GB of RAM. Saves on power as well. Price will be $1100 or $275 per node.
-
You need to add some drives as well but other than that you are set.
Put this baby in a 2U colo space and have complete remote control of every server over IPMI.
-
I think it's pretty awesome.
Server like these are what the big boys buy and they get thousands of them at a time. So they haven't spent their life in a hot closet at some SMB. They've spent their life in a big datacenter somewhere with proper cooling and no dust. It was replaces when it was time to refresh the hardware, not when it became unreliable and unusable.
-
The reason they are cheap is that they show up in large volumes at the same time and usually they are without CPU and RAM. Most hobbyists and a lot of people in IT are not familiar with multi-node servers so they don't know exactly what it is and how to use it. Some confuse them with blade servers as well.
-
How loud is such a setup? I have a single Dell T420 for my lab in my apartment, since it's generally quiet. I've been thinking about how I could expand to having more physical hosts.
-
Enjoy your $500+ power bill
-
This doesn't seem wife approved
-
Also how do you plan to handle cooling?
-
@Pete-S said in Building your own lab:
The reason they are cheap is that they show up in large volumes at the same time and usually they are without CPU and RAM. Most hobbyists and a lot of people in IT are not familiar with multi-node servers so they don't know exactly what it is and how to use it. Some confuse them with blade servers as well.
True, I was not aware of multi-node servers.
Whether those 4 servers are individual, except cooling system and server chassis?
Why there are only two power supplies for 4 nodes?
How it would be if require any maintenance on 1 node? you can manage individual or impact other 3 nodes in this process?
-
Not to be a downer - in these times- who has an extra $1,300? I sincerely hope that all have kept their employment as I have - $1,300 will go a long way in repairs on a vehicle I can’t afford to replace.
Having a home lab is a great tool but just doesn’t seem practical to invest in right now.
-
@openit said in Building your own lab:
True, I was not aware of multi-node servers.
They are called blades. This is a tiny blade unit. Very common in 2, 4, 8 and 16 node varieties.
-
@gjacobse said in Building your own lab:
Not to be a downer - in these times- who has an extra $1,300? I sincerely hope that all have kept their employment as I have - $1,300 will go a long way in repairs on a vehicle I can’t afford to replace.
Having a home lab is a great tool but just doesn’t seem practical to invest in right now.
It's always good to invest. In fact, investing when times are tough often leads to the lowest investment cost and the highest returns. Holding back investments due to uncertainty is often a cause for the greatest losses.
It is specifically because times are uncertain that embracing growth and preparing to be as ready as possible for the job market matters more than normal.
-
@openit said in Building your own lab:
Whether those 4 servers are individual, except cooling system and server chassis?
Yes, the servers are physically discrete logically. They share power and chassis, but they can't "see" each other.
-
@openit said in Building your own lab:
Why there are only two power supplies for 4 nodes?
Because that's where the big savings come from - not needing eight power supplies to get redundancy, you only need two. That reduces power consumption and cost and space, dramatically. If it wasn't for the shared power and chassis, blades would have no purpose to exist.
-
@openit said in Building your own lab:
How it would be if require any maintenance on 1 node? you can manage individual or impact other 3 nodes in this process?
That depends on the specific unit, but typically nodes are hot swap.
-
@VoIP_n00b said in Building your own lab:
Enjoy your $500+ power bill
Labs aren't cheap, but they are a huge, huge deal in career advancement.
-
@VoIP_n00b said in Building your own lab:
Also how do you plan to handle cooling?
If you have an air conditioned house normal people won't have cooling issues with lab at home. It's not that big and you can power down when you have to.
If you live in a climate with a basement, then it's even easier.
-
I think most people separate multi-node servers from blade servers because there are some minor differences. Blades are multi-node too of course but also shares management, sometimes networking and other things in the blade chassis that the multi-node servers do not.
Supermicro for example puts these products in two different categories.
I think there is also a difference in how you buy them. Multi-node servers comes with all nodes while blade servers are separated into chassis and blades.
-
What node has access to the disks or is each node responsible for 3 drives?
Can you assign one node all the storage and the remaining 3 be just compute? -
@scottalanmiller said in Building your own lab:
@gjacobse said in Building your own lab:
Not to be a downer - in these times- who has an extra $1,300? I sincerely hope that all have kept their employment as I have - $1,300 will go a long way in repairs on a vehicle I can’t afford to replace.
Having a home lab is a great tool but just doesn’t seem practical to invest in right now.
It's always good to invest. In fact, investing when times are tough often leads to the lowest investment cost and the highest returns. Holding back investments due to uncertainty is often a cause for the greatest losses.
It is specifically because times are uncertain that embracing growth and preparing to be as ready as possible for the job market matters more than normal.
I can understand this principle, however tell that to my mechanic.
Truck needs a new power steering pump and a boat load of other things. Price paid for having to drive for work and having a fifteen year old vehicle.
