Help - Setting up raid on Debian

[rv8000]

I picked up some cheap parts and 2x6tb HDDs to setup in raid for redundancy as a home server/plex server using Debian.

I’ve had some success creating the raid array through terminal/mdadm, but I’ve not been able to get the array to persist through restarts as I can’t edit the auto mount with the /etc/fstab command as it says “you don’t have permission” (I’ve already entered the root user/password in terminal).

After a few more attempts and trying other commands it sort of looked like I got into a file for /etc/fstab, but guide documentation is so poor or have no ss to the point I’d rather not break things because I’ve no good idea where to put this “line of code” they reference.

So… before going any further:

1) Is there a GUI/software for debian I can get that simplifies raid creation?
2) Whats the best raid format (read a few things about ZFS, lots of conflicting information)? I just want simple redundancy here…
3) What partition type for 6tb and to be viewable as a network location to different OS/devices?
4) Once the raid array is setup, how can it be mounted in a way that I can see it as a default location in the file explorer (documents, music, etc…) represented by the partition name (or is the raid array name something other than “md0” to solve that, if going through mdadm)?

This as be the most mind numbingly frustrating software experience in my life, to say the least… Assistance would be much appreciated.

 

[_roman_]

Do you have a hardware raid controller? yes? Or a fake raid of the mainboard?

 

[rv8000]

Do you have a hardware raid controller? yes? Or a fake raid of the mainboard?

Id assume it’s software raid through the mb

 

[qxp]

I assume you followed directions in the link below. Can you check that mdadm.conf file exists? If yes, check that mdadm-raid service is enabled - this is what starts the raid on boot.

SoftwareRAID - Debian Wiki

wiki.debian.org

 

[Deleted member 245263]

i suggest using 'micro' file editor instead of nano or vim (if you are not accustomed with vim)

it asks if you want to save the file with sudo - admin privileges

obv works in text terminal

 

[Aquinus]

I have the very setup you're looking at doing on my desktop when I boot in Ubuntu. I use a NVMe drive for the OS, and a MDADM RAID-0 of two SATA SSDs and a RAID-5 of 4x1TB WD Blacks. You're going down the right path thinking that you need to update `/etc/fstab` once the RAID and partitions have been created. You'll want to add those entries using the bulk device uuid.

1) Is there a GUI/software for debian I can get that simplifies raid creation?

I would suggest doing everything in the terminal directly with MDADM. Debian tends to be pretty sparse when it comes to GUI tooling out of the box and I'm not aware of any good tooling for this.

2) Whats the best raid format (read a few things about ZFS, lots of conflicting information)? I just want simple redundancy here…

I just use EXT4 for everything and haven't had a problem with it, but that's the file system. RAID mode or level is something completely different. I use RAID-5 for my redundant array, which is formatted with EXT4.

3) What partition type for 6tb and to be viewable as a network location to different OS/devices?

Assuming you'll be using Samba, the file system and type of partition it is does not matter. Use whatever floats your boat.

4) Once the raid array is setup, how can it be mounted in a way that I can see it as a default location in the file explorer (documents, music, etc…) represented by the partition name (or is the raid array name something other than “md0” to solve that, if going through mdadm)?

To clarify, are you asking if it's possible to have `/home` be mounted to one of your RAID partitions? If so, I'm not entirely sure how that'd be done after the OS is already installed. I'd imagine that it would involve starting the computer with a live usb, copying the existing files from `/home` to the new RAID partition, and updating your fstab to mount `/home` from that partition immediately after root; `/`.

 

[rv8000]

i suggest using 'micro' file editor instead of nano or vim (if you are not accustomed with vim)

it asks if you want to save the file with sudo - admin privileges

obv works in text terminal



View attachment 398832

View attachment 398831

The steps I have issues with are formatting the partition through terminal; it’s my understanding that EXT4 is only visible by Linux based systems. If I want to see the raid 1 array on a windows based device as a network location, the format has to be something other than ext4, fat (limited to 2tb), or another partition type correct? So “sudo mkfs.ext4 /dev/md0” would be the incorrect choice if I’m understanding this.

And the other issue was when attempting to edit /etc/fstab the guide (happened to be another, cant link atm) did not explicitly say where to add the line of text and or save the file.

I’ve never worked with Linux anything so this is rather new/painful.

I have the very setup you're looking at doing on my desktop when I boot in Ubuntu. I use a NVMe drive for the OS, and a MDADM RAID-0 of two SATA SSDs and a RAID-5 of 4x1TB WD Blacks. You're going down the right path thinking that you need to update `/etc/fstab` once the RAID and partitions have been created. You'll want to add those entries using the bulk device uuid.

I would suggest doing everything in the terminal directly with MDADM. Debian tends to be pretty sparse when it comes to GUI tooling out of the box and I'm not aware of any good tooling for this.

I just use EXT4 for everything and haven't had a problem with it, but that's the file system. RAID mode or level is something completely different. I use RAID-5 for my redundant array, which is formatted with EXT4.

Assuming you'll be using Samba, the file system and type of partition it is does not matter. Use whatever floats your boat.

To clarify, are you asking if it's possible to have `/home` be mounted to one of your RAID partitions? If so, I'm not entirely sure how that'd be done after the OS is already installed. I'd imagine that it would involve starting the computer with a live usb, copying the existing files from `/home` to the new RAID partition, and updating your fstab to mount `/home` from that partition immediately after root; `/`.

Mentioned before, ext4 is only visible by linux os yes? I dont care what format is used in the end, just which can be see by both windows and linux as a network location.

If samba is the remote access software, yes. In attempting to get everything rolling I’ve been doing everything directly on the server atm.

I’d just like to have the raid array visible in the file explorer viewer, think like how windows would see a newly partitioned drive in file explorer (c drive, d drive, e drive and so on); the times I had successfully mounted I had to go to “other locations” to find the drive, this just seems weird as figured it would behave more like windows seeing it as a new drive and default location once setup. I may be wording this incorrectly, apologies as yesterday was the first few hours doing anything with debian/linux.

*follow up edit

If I were to create the mount point “/mnt/Brains/“

Would debian file explorer see it as a folder named “Brains” under other locations?

 

[Aquinus]

Mentioned before, ext4 is only visible by linux os yes? I dont care what format is used in the end, just which can be see by both windows and linux as a network location.

EXT4 is only readable by Linux on the host machine, yes, BUT if you're planning on this machine running Linux all the time and sharing the data on it via a network share, then the filesystem doesn't matter for it to be accessible in Windows as Samba will handle all those interactions with the local filesystem.

I’d just like to have the raid array visible in the file explorer viewer, think like how windows would see a newly partitioned drive in file explorer (c drive, d drive, e drive and so on); the times I had successfully mounted I had to go to “other locations” to find the drive, this just seems weird as figured it would behave more like windows seeing it as a new drive and default location once setup. I may be wording this incorrectly, apologies as yesterday was the first few hours doing anything with debian/linux.

I'm not on my Linux machine right now, but off the top of my head the way I added it to the file browser in Ubuntu is that I opened `/mnt` and dragged the mounted directory to the left hand navigation in the Windows to create the shortcut. This could be different in Debian and I also could be remembering incorrectly. Later on I can start that machine and take a look.

 

[qxp]

Just to add to the discussion - you only need to be worried about filesystem being readable by Linux, because a network export does NOT export raw disk, but just passes requests about files. So Samba would let Windows see any files on any Linux filesystem you gave samba access to.

I highly recommend "vim". Make sure to install the full version with "apt install vim" - the one included by default is minimized for size and omits useful features.

The great advantage of vim is that you need to enter a command to change anything. This makes editing system configuration files much safer.

vim has many features, but for configuration file editing it is quite simple:

Open file with "vim file.conf". Use arrows to put cursor where you want. Press "i" to insert text. Press "Escape" when you are done editing. Press "x" to delete a single character. Press "dd" to delete a line.

When you are sure the file looks good, save and exit by typing ":wq[ENTER]".

 

[rv8000]

I’ll take a look at these options when I’m back home in the evening and post any follow up questions/ss. Thank you for the help so far.

 

[redeye]

Just to add to the discussion - you only need to be worried about filesystem being readable by Linux, because a network export does NOT export raw disk, but just passes requests about files. So Samba would let Windows see any files on any Linux filesystem you gave samba access to.

I highly recommend "vim". Make sure to install the full version with "apt install vim" - the one included by default is minimized for size and omits useful features.

The great advantage of vim is that you need to enter a command to change anything. This makes editing system configuration files much safer.

vim has many features, but for configuration file editing it is quite simple:

Open file with "vim file.conf". Use arrows to put cursor where you want. Press "i" to insert text. Press "Escape" when you are done editing. Press "x" to delete a single character. Press "dd" to delete a line.

When you are sure the file looks good, save and exit by typing ":wq[ENTER]".

more important to press (esc), to get out of any mode… and does not hurt to (esc) … :q! to exit, without saving. or “ZZ”

while it’s worth the effort to learn vim (no need for the mouse!) because it’s 1337uber powerful, nano is more like the way people who are used to windows edit. most amazing thing at i have used is the column selection ability in vim. so easy to comment out a block of text.

i would always use nano, but i was installing “PMSS” on a server i rented, then was thrown into vim… what hell?!… how do i get out, i’m trapped… no problem google it. solved. then i needed to the crap that “PMSS” left behind, because it does not wotk on debian 12, (so many cron jobs) … so had to comment out 10 files. was not going to “#” each line to much work, so i googled adding a character to a text line linux “ and got instructions for vim’s column mode… so thanks to the unknown programmer that 50 years ago needed to do that and added the feature to (vi) the precursor to VIM.
(and only real programmers use VIM , lol)

on topic, using raid 0, is going to doom your data…and software raid is a pain…

 

[rv8000]

more important to press (esc), to get out of any mode… and does not hurt to (esc) … :q! to exit, without saving. or “ZZ”

while it’s worth the effort to learn vim (no need for the mouse!) because it’s 1337uber powerful, nano is more like the way people who are used to windows edit. most amazing thing at i have used is the column selection ability in vim. so easy to comment out a block of text.

i would always use nano, but i was installing “PMSS” on a server i rented, then was thrown into vim… what hell?!… how do i get out, i’m trapped… no problem google it. solved. then i needed to the crap that “PMSS” left behind, because it does not wotk on debian 12, (so many cron jobs) … so had to comment out 10 files. was not going to “#” each line to much work, so i googled adding a character to a text line linux “ and got instructions for vim’s column mode… so thanks to the unknown programmer that 50 years ago needed to do that and added the feature to (vi) the precursor to VIM.
(and only real programmers use VIM , lol)

on topic, using raid 0, is going to doom your data…and software raid is a pain…

I would be setting up raid 1 via mdadm (software raid I assume).

 

[redeye]

I would be setting up raid 1 via mdadm (software raid I assume).

that is great, but raid is not a “backup”… (raid 1 is for more protecting against hardware failure. IMO)

 

[qxp]

that is great, but raid is not a “backup”… (raid 1 is for more protecting against hardware failure. IMO)

And increasing performance - you get aggregate bandwidth from all the drives.

Btw, another vim feature is regular expressions support. For example, if you needed to add "#" to the beginning of every line you could have issued :%s/^/#/g

You can also tell vim to repeat a command specified number of times. For example, typing "10dd" will delete 10 lines.

 

[_roman_]

I picked up some cheap parts and 2x6tb HDDs to setup in raid for redundancy as a home server/plex server using Debian.

I assume you just want more checksums, right?

I would look into lvm2. e.g mirror option. //// Those software fake raids have lots of topics and guides anyway.

Samba - whatever - was already mentioned.

I have backups in ext4. But I use lvm2 -> LUKS -> BTRFS for /

i would not space waste my discs with ext4. I do use lvm2/luks/ext4 for backups. I do use ext4 for my trash download HDD - which gets checksummed anyway before the data is being used.

There are some other file systems which use the disk space more efficient. For a network ressource I would not use a low desktop idle setting of 3, but a much higher setting.

[  148.203220] BTRFS info (device dm-1 state M): use zstd compression, level 3

I’ve had some success creating the raid array through terminal/mdadm, but I’ve not been able to get the array to persist through restarts as I can’t edit the auto mount with the /etc/fstab command as it says “you don’t have permission” (I’ve already entered the root user/password in terminal).

I'm not sure what you are doing.

I moved this gentoo installation from 2006 several times. File System changes, encryption, lvm, different storage.

I would check first how you boot - the hole boot process
e.g. lazy approach for myself
zgrep CONFIG_CMDLINE\= /proc/config.gz
I would check which user does those changes - Sudo is very, very, very, very, very bad
I would check again how you boot and if you have used properly addressing. I think these days they call it UUID, Proper settings of /etc/fstab // kernel // bootloader // kernel build in cmd_line
e.g man lsblk
Sometimes you need to work with a live environment. that would also solve your issue - no permissions. Many of those livecds i used over the years had only one user - root
if you typed more than once sudo or su - you have a problem with the user typing the commands!

 

[rv8000]

that is great, but raid is not a “backup”… (raid 1 is for more protecting against hardware failure. IMO)

Backups will be handled with another drive. I’m not super concerned with the data as the majority will be music (already backed up in multiple locations) and video files (I’ll back these up as I populate the raid array when adding media) for plex.

Raid 1 is purely for the hardware failure aspect/redudancy.

It seems I’ve opened a can of worms. This is meant to be a very, very simple plex server, that is basically it.

 

[Deleted member 245263]

making persistent across reboots

if you will be sharing this via samba or NFS you dont care what file system is this

I assume you followed directions in the link below. Can you check that mdadm.conf file exists? If yes, check that mdadm-raid service is enabled - this is what starts the raid on boot.

SoftwareRAID - Debian Wiki

wiki.debian.org

 

[Aquinus]

i would not space waste my discs with ext4.

You mind expanding upon that? As I understand it, btrfs doesn't support online deduplication at current unless you turn on some experimental features. So unless the OP is copying files a lot, I don't see ext4's lack of deduplication as being a disadvantage. For the sake of stability and keeping the setup simple is why I suggest ext4. The OP isn't looking for something exotic.

It seems I’ve opened a can of worms. This is meant to be a very, very simple plex server, that is basically it.

Nah. Everybody just has opinions on how things should be done. I think what you've done already is fine.

With that said, a lot has been said. Are you still stuck?

 

[BADCAP]

I logged in to leave my nybble-

I get the impression that your goal is to get something like mirrored RAID served up over a network share in addition to running plex, and not necessarily to take sysadmin101 in the process.

If I'm wrong in that assessment (and you want to learn along the way), then you might want to start by familiarizing yourself with the man pages for the various programs you'd be using on your adventure:

man fstab
man samba
man mdadm (or, better yet, man btrfs)

If you're over it and want something that will work, is quick to set up, has a web interface, is free (as in liberty and beer), then I'd suggest you take a look at openmediavault (which is based on Debian). Start with the documentation: https://docs.openmediavault.org/en/latest/index.html

For a simple mirror, I'd recommend you consider btrfs over md software raid, though that's just my opinion. Benefits of btrfs include checksuming (better data integrity), better tools (for managing the pool and performing maintenance like scrubs and replacing faulty disks), and being a copy on write filesystem (think more resilient). Downsides are that RAID levels outside of 0 and 1 are still unstable. I think btrfs is now the default option in OMV now for simple arrays, like a mirror, and I'm fairly sure that the developer has expressed his intent to continue to lean more towards btrfs in future updates/releases. I'm not sure though since I use a more customized setup with ZFS and don't check the forums there too often.

The filesystem used doesn't really impact your ability to access shares over samba (with the exception of more exotic features that you're unlikely to use like version history and more granular permissions). You'll be able to access a samba share from a windows machine (or mac or a different distro of linux) regardless of the filesystem used on the server.

The OMV fourms and documentation are very helpful, though I would strongly recommend you search/read before posting a question as it's probably already been answered. I'm pretty sure there's even a guide on how to get jellyfin/plex running in docker or kubernetes somewhere on the forums. Most of your questions for the basic setup (outside of plex) should be answered by the docs.

Consider supporting the dev if you end up using and liking the software.

GLHF

 

[rv8000]

You mind expanding upon that? As I understand it, btrfs doesn't support online deduplication at current unless you turn on some experimental features. So unless the OP is copying files a lot, I don't see ext4's lack of deduplication as being a disadvantage. For the sake of stability and keeping the setup simple is why I suggest ext4. The OP isn't looking for something exotic.

Nah. Everybody just has opinions on how things should be done. I think what you've done already is fine.

With that said, a lot has been said. Are you still stuck?

I’ve gotten the raid array to persist through restarts but I’m currently having two issues.

1) Navigating to the location of the file system seems odd (mnt point was set to /mnt/desiredfoldername) and I go through the Debian GNU/linux/mnt location to find it.

If I want it to appear under the “home” tab in file browser, would I have to set the mnt point to /home/desiredfoldername?

2) I cannot add folders inside the main one. I think the permissions are set to the root user exclusively. The debian install had me create a user account, so I’m unsure what the best way to remedy this is. If the server is always on and logged in to the user account created, do I simply have to enable edit permissions for that user?

 

[qxp]

I’ve gotten the raid array to persist through restarts but I’m currently having two issues.

1) Navigating to the location of the file system seems odd (mnt point was set to /mnt/desiredfoldername) and I go through the Debian GNU/linux/mnt location to find it.

If I want it to appear under the “home” tab in file browser, would I have to set the mnt point to /home/desiredfoldername?

2) I cannot add folders inside the main one. I think the permissions are set to the root user exclusively. The debian install had me create a user account, so I’m unsure what the best way to remedy this is. If the server is always on and logged in to the user account created, do I simply have to enable edit permissions for that user?

I would suggest to do "sudo mkdir /mnt/desiredfoldername/$USER" and then "sudo chown $USER /mnt/desiredfoldername/$USER", and finally "ln -s /mnt/desiredfoldername/$USER /home/$USER/raid".

This makes a user-specific folder that is hosted on your mount.

If you want a shared directory between several users you do the same but create it in /mnt/desiredfoldername/$USER and use sticky bits to make files accessible to users in a group.

 

[Deleted member 245263]

I’ve gotten the raid array to persist through restarts but I’m currently having two issues.

1) Navigating to the location of the file system seems odd (mnt point was set to /mnt/desiredfoldername) and I go through the Debian GNU/linux/mnt location to find it.

If I want it to appear under the “home” tab in file browser, would I have to set the mnt point to /home/desiredfoldername?

the folders in /home/'USERNAME' are created when you create a user - there are stored settings for that user if you logged to that account. So if you now want to mount other partition in place of this folder you basically need to move the files first (or this user will have his settings disappear once you mount). OR You can also just create a link in your file browser to the mount point - like for example /mnt/h, /mnt/g, Projects, libraries etc., if this is desired GUI behaviour.

2) I cannot add folders inside the main one. I think the permissions are set to the root user exclusively. The debian install had me create a user account, so I’m unsure what the best way to remedy this is. If the server is always on and logged in to the user account created, do I simply have to enable edit permissions for that user?

yes, if its ext4 just change permissions to the mount point. If its non-linux filesystem you set permissions at mount time.