HPC Cluster planning workflow?

Posted by Veronica on Server Fault See other posts from Server Fault or by Veronica
Published on 2012-10-22T05:27:15Z Indexed on 2012/10/22 11:04 UTC
Read the original article Hit count: 282

Filed under:

hpc

|

capacity-planning

|

planning

|

Workflow

|

rocks

After three days of intensive Google searching, I have not found any high-level workflow of how to build a low profile - cheap - computing cluster (we are not interested in HA yet). This is just a front-end plus a node for now.

We want to start small with rockscluster, provide a web-based server for offering services, and then add nodes as our budget increases. We're small company, so we haven't enough human resources to implement it smoothly. Here are some facts about our environment:

Our hardware is not constant (we will add nodes).
Our workload will vary (in the order from 200Mb - 1Tb)
Our software will change (scientific applications for data mining)

Do you know any visual workflow, worksheet, chart, describing the general necessary steps to begin our cluster planning?

© Server Fault or respective owner

Related posts about hpc

Windows HPC Server links

as seen on Daniel Moth - Search for 'Daniel Moth'
I've already described how to setup a Windows HPC Server for development. Before you dive into developing for the cluster, if you are new to this it is probably a good idea to learn the basics by reading some overview material. Below is a list of links.Direct Links to Windows HPC content1. Windows… >>> More
Azure Grid Computing - Worker Roles as HPC Compute Nodes

as seen on Geeks with Blogs - Search for 'Geeks with Blogs'
Overview · With HPC 2008 R2 SP1 You can add Azure worker roles as compute nodes in a local Windows HPC Server cluster. · The subscription for Windows Azure like any other Azure Service - charged for the time that the role instances are available, as well as for the compute and storage… >>> More
Penguin's HPC Waddle

as seen on Internet.com - Search for 'Internet.com'
Server Snapshot: Penguin Computing has always, as its name implies, focused on developing best practices for Linux-based systems, software and services, particularly in the HPC space. >>> More
Penguin's HPC Waddle

as seen on Internet.com - Search for 'Internet.com'
Server Snapshot: Penguin Computing has always, as its name implies, focused on developing best practices for Linux-based systems, software and services, particularly in the HPC space. >>> More
Appro: Bringing HPC to the Mainstream Enterprise

as seen on Internet.com - Search for 'Internet.com'
Server Snapshot: Appro has always been about HPC. But HPC is not what it used to be, and Appro has changed to meet the needs of more mainstream enterprises with its revamped product lines built around a Scalable Unit architecture. >>> More

Related posts about capacity-planning

Throughput; capacity planning help for C10K like design

as seen on Server Fault - Search for 'Server Fault'
I am designing a network service in which clients connect and stay connected -- the model is not far off from IRC less the s2s connections. I could use some help understanding how to do capacity planning, in particular with the system resource costs associated with handling messages from/to clients… >>> More
Developer Capacity Planning

as seen on Stack Overflow - Search for 'Stack Overflow'
What product or tool do you use to do Resource Capacity planning for your development team? Currently managing a large group and trying to provide visibility into my resource availability. I am creating this in MS project, but was curious if there was a better tool/solution based off your experience. >>> More
T-SQL Tuesday - IO capacity planning

as seen on SQL Blog - Search for 'SQL Blog'
This post is my contribution to Adam Machanic's T-SQL Tuesday #004 , hosted this time by Mike Walsh . Being applicative DBA, I usually don't take part in discussions which storage to buy or how to configure it. My interaction with IO is usually via PerfMon. When somebody calls me asking why everything… >>> More
Capacity Planning IIS6

as seen on Server Fault - Search for 'Server Fault'
Hello, I am planning to host 5000 site with separate application pool. Based on my estimate there are around 1600- 2000 worker process will be running on the server. So my question is: Is is possible to host 5000 sites on a server with IIS 6. Server Configuration: 16 Core @ 2.27 Ghz CPU 8 GB… >>> More
Maximum number of files in one ext3 directory while still getting acceptable performance?

as seen on Server Fault - Search for 'Server Fault'
I have an application writing to an ext3 directory which over time has grown to roughly three million files. Needless to say, reading the file listing of this directory is unbearably slow. I don't blame ext3. The proper solution would have been to let the application code write to sub-directories… >>> More