1. Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.

Strange idea... possible to cluster computers on a network for general processing?

Discussion in 'General Hardware' started by hat, Jul 20, 2014.

  1. hat

    hat Enthusiast

    Joined:
    Nov 20, 2006
    Messages:
    17,197 (5.46/day)
    Thanks Received:
    2,198
    Location:
    Ohio
    Would it be possible to hook together a number of widely different computers (such as my Q6600 system, this old Pentium 4 sitting over here, and this one AMD desktop sitting over here), and link them up to run general tasks and assist my main system? Possible applications... handbrake/convertx encoding, improved performance in games? Maybe by way of a "Host" program installed on each system, which I would then use on the main system to target and launch whichever program I wanted to run with the assistance of the others (e.g. Handbrake)?
     
    Crunching for Team TPU More than 25k PPD
  2. Devon68

    Devon68

    Joined:
    Jan 25, 2014
    Messages:
    1,136 (2.16/day)
    Thanks Received:
    589
  3. hat

    hat Enthusiast

    Joined:
    Nov 20, 2006
    Messages:
    17,197 (5.46/day)
    Thanks Received:
    2,198
    Location:
    Ohio
    Unfortunately he doesn't describe what he used them for or how they were linked or anything... I tried posting maybe he'll get back to it.
     
    Crunching for Team TPU More than 25k PPD
  4. 95Viper

    95Viper

    Joined:
    Oct 12, 2008
    Messages:
    4,691 (1.91/day)
    Thanks Received:
    1,796
    Location:
    στο άλφα έως ωμέγα
    Not so strange...

    Tech Article - DIY Supercomputing: How to Build a Small Windows HPC Cluster

    Building Your Own Super Computer

    Building your own supercomputer using Ubuntu and Kerrighed


    Quote from here --> How to Hook Up Several Computers to Make a Supercomputer

     
    Last edited: Jul 20, 2014
    Devon68, T-Bob and FordGT90Concept say thanks.
  5. Vario

    Vario

    Joined:
    Oct 21, 2005
    Messages:
    3,143 (0.89/day)
    Thanks Received:
    1,052
    beowulf cluster maybe?
     
  6. Aquinus

    Aquinus Resident Wat-man

    Joined:
    Jan 28, 2012
    Messages:
    7,649 (6.10/day)
    Thanks Received:
    3,020
    Location:
    Concord, NH
    I don't think what you want to do exactly exists. You can't easily divvy up tasks the way that you're suggesting. If you had a particular application in mind, you can always develop something yourself but most applications that take advantage of networked HPC are similar to GPGPU tasks. Some applications like Transmission support remote management, but that's not "clustered".
     
  7. yogurt_21

    yogurt_21

    Joined:
    Feb 18, 2006
    Messages:
    4,487 (1.31/day)
    Thanks Received:
    607
    Location:
    AZ
    ^ I think you hit on it when you said it doesn't "exactly exist" visualization and things like xenapp do a piece of of it, clustering and etc could do another. But nothing will essentially allow you to take all of your rigs power and combine it together for standard day to day tasks. Video encoding and converting might actually be slower as the HD's and networking might not be fast enough to keep up. Gaming certainly would be slower.

    For me I'd look more into programs that do the same thing handbrake does only better/faster. The best video encoder/transcoder I've found is FFmpeg. Simple command line based but very fast and robust.

    https://www.ffmpeg.org/

    Not only do we use it at work to re-encode out mp4's for student lectures, but the NewTek Tricasters we use for filming use it to film to mp4 as well.
     
  8. Shambles1980

    Joined:
    May 3, 2014
    Messages:
    541 (1.27/day)
    Thanks Received:
    102
    thought all the nodes in a cluster had to use the same hardware and software. thats why i never bothered to try and set one up.
     
  9. hat

    hat Enthusiast

    Joined:
    Nov 20, 2006
    Messages:
    17,197 (5.46/day)
    Thanks Received:
    2,198
    Location:
    Ohio
    Sounds very limited and with specific uses.
     
    Crunching for Team TPU More than 25k PPD
  10. yogurt_21

    yogurt_21

    Joined:
    Feb 18, 2006
    Messages:
    4,487 (1.31/day)
    Thanks Received:
    607
    Location:
    AZ
    ? really.

    The first description on their site
    It's not an audio studio perhaps, but there's almost nothing it hasn't been able to do with video. And it still does have a crap ton of options for audio.
     
  11. Sasqui

    Sasqui

    Joined:
    Dec 6, 2005
    Messages:
    7,976 (2.28/day)
    Thanks Received:
    1,669
    Location:
    Manchester, NH
    Used to work with Incredibuild, https://www.incredibuild.com/ . Biggest performance feature is a distributed compiler. Each client computer needed to have it installed where it could take advantage of idle CPU time on remote computers to compile Visual Studio projects. Cut compiling time by 10x (at least).

    So long story short, without an OS or generic program to break up and distribute tasks, you probably need specific software that does the parallel task management and distribution to manage whatever it is you're trying to do faster.
     
  12. Jizzler

    Jizzler

    Joined:
    Aug 10, 2007
    Messages:
    3,591 (1.24/day)
    Thanks Received:
    700
    Location:
    Geneva, FL, USA
    Not exactly what you're looking for, just throwing out ideas.

    There is software (the name escapes me at the moment) that acts as a KVM of sorts. Say you have 4 computers, you set them up like you would multiple displays so when you move your mouse from one screen to another, you are sending input to that specific computer. Start Handbrake on one, friends twitch stream on another, etc.

    It's similar to if you Remote Desktop (Teamviewer, etc) to each machine but with some added benefits, as long as you have a monitor per computer.


    You see that a lot because it's easier to buy and build in bulk, but a cluster can be designed in various ways. Sometimes it's best to be identical across the board, and other times it's good to have nodes tailored for different tasks. Other times, it's just because there's some hardware laying around and you design around what you have ;)
     
  13. Frick

    Frick Fishfaced Nincompoop

    Joined:
    Feb 27, 2006
    Messages:
    11,479 (3.36/day)
    Thanks Received:
    2,832
    That's clusters for you.

    Now this is the territory of cloud computing.
     
    Aquinus says thanks.
  14. Aquinus

    Aquinus Resident Wat-man

    Joined:
    Jan 28, 2012
    Messages:
    7,649 (6.10/day)
    Thanks Received:
    3,020
    Location:
    Concord, NH
    I find is amusing how people think code can be made multi-threaded without any issue and now people think cloud computing is easy as pie. Sorry to break it to you people, but concurrent programming isn't easy and isn't always possible. Cloud configurations are much harder to manage than multi-threaded ones, which is why you see them even less of them unless the applications demands performance (real time applications.) If you're not a SysAdmin or a Developer with a good reason for doing it, you probably shouldn't be considering it.
     
    yogurt_21 says thanks.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guest)

Share This Page