ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Solved Using Rclone from a server to backup multiple volumes from different SSH hosts

    IT Discussion
    rclone b2 backup automation users
    4
    20
    1.2k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • DustinB3403D
      DustinB3403
      last edited by

      One option I considered is just installing Cyberduck on the workstations and having the user do the backup.

      Of course that just won't be done and thus is off the table.

      1 Reply Last reply Reply Quote 0
      • M
        marcinozga @DustinB3403
        last edited by marcinozga

        @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

        @marcinozga said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

        @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

        @marcinozga said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

        @DustinB3403 Install rclone on each workstation and setup cron jobs on each machine. Brew and Ansible make this trivial. Do you really need Fedora Server in this setup?

        This I considered, but would involve tinkering with each workstation.

        That's why I mentioned Ansible. Your tinkering is limited to workstation that runs playbooks.

        That's not at all accurate, I would have to install rclone on every workstation, have custom cron jobs on every workstation and then would have to validate that every workstation sync'd.

        And all that you can do with Ansible.
        Install rclone? https://docs.ansible.com/ansible/latest/modules/homebrew_module.html#homebrew-module
        Setup cron? https://docs.ansible.com/ansible/latest/modules/cron_module.html#cron-module

        1 Reply Last reply Reply Quote 0
        • M
          marcinozga @DustinB3403
          last edited by marcinozga

          @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

          In addition to that, I would have to enable cron on every apple workstation, which while doable, is a pain in the rear should I ever need to make any changes to the backup operation.

          Centrally managing this makes way more sense rather than the ad hoc approach.

          And since when has cron been disabled in macOS?

          DustinB3403D 1 Reply Last reply Reply Quote 0
          • DustinB3403D
            DustinB3403 @marcinozga
            last edited by

            @marcinozga said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

            @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

            In addition to that, I would have to enable cron on every apple workstation, which while doable, is a pain in the rear should I ever need to make any changes to the backup operation.

            Centrally managing this makes way more sense rather than the ad hoc approach.

            And since when has cron been disabled in macOS?

            For quite some time, it's still there, but you have to enable it, grant it root access etc.

            M 1 Reply Last reply Reply Quote 0
            • M
              marcinozga @DustinB3403
              last edited by

              @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

              @marcinozga said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

              @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

              In addition to that, I would have to enable cron on every apple workstation, which while doable, is a pain in the rear should I ever need to make any changes to the backup operation.

              Centrally managing this makes way more sense rather than the ad hoc approach.

              And since when has cron been disabled in macOS?

              For quite some time, it's still there, but you have to enable it, grant it root access etc.

              That's simply not true. You don't have to enable it. And you don't have to give it root access unless you're writing somewhere you shouldn't be in first place.

              1 Reply Last reply Reply Quote 0
              • DustinB3403D
                DustinB3403
                last edited by

                Anyways I figured this out.

                Having multiple rclone sources like
                https://i.imgur.com/SKtlxrr.png

                and running

                b9469d45-8af9-4407-b185-7105a79a7557-image.png

                Works

                dafyreD 1 Reply Last reply Reply Quote 2
                • dafyreD
                  dafyre @DustinB3403
                  last edited by

                  @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

                  Anyways I figured this out.

                  Having multiple rclone sources like
                  https://i.imgur.com/SKtlxrr.png

                  and running

                  b9469d45-8af9-4407-b185-7105a79a7557-image.png

                  Works

                  The /Volumes/<path to external> would change for each system, wouldn't it?

                  Could you back up to b2-rclone:/<username>/external ?

                  DustinB3403D 1 Reply Last reply Reply Quote 0
                  • DustinB3403D
                    DustinB3403 @dafyre
                    last edited by

                    @dafyre said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

                    @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

                    Anyways I figured this out.

                    Having multiple rclone sources like
                    https://i.imgur.com/SKtlxrr.png

                    and running

                    b9469d45-8af9-4407-b185-7105a79a7557-image.png

                    Works

                    The /Volumes/<path to external> would change for each system, wouldn't it?

                    Could you back up to b2-rclone:/<username>/external ?

                    Possibly, but what I was thinking I would do is simply connect to the administrative user on each of these workstations, hit the volume that way. The Volume name may change (as spaces make everything more difficult). But yeah I think this'll work.

                    1 Reply Last reply Reply Quote 0
                    • DustinB3403D
                      DustinB3403
                      last edited by

                      To clarify for anyone else in the future, you need to have multiple apple-# sftp connections.

                      The b2 connections could also be custom as you can create multiple buckets and simply connect to that bucket via rclone.

                      1 Reply Last reply Reply Quote 0
                      • DustinB3403D
                        DustinB3403
                        last edited by

                        This is what the b2-rclone remote setting looks like for my needs, obviously this could be adjusted with the advanced configuration menu.

                        https://i.imgur.com/sj858rr.png

                        And this is the Apple workstation

                        https://i.imgur.com/jGytpzZ.png

                        1 Reply Last reply Reply Quote 0
                        • DustinB3403D
                          DustinB3403
                          last edited by

                          Ideally when this is all said and done, what I think would make the most sense is to create multiple buckets for each of these apple workstations.

                          So apple-bucket-1, apple-bucket-2 and so on.

                          From there, simply have cron run the sync commands for each of these and have it run hourly.

                          1 Reply Last reply Reply Quote 0
                          • DustinB3403D
                            DustinB3403
                            last edited by DustinB3403

                            Worth noting on this that while sync works, copy is the much safer option.

                            So

                            rclone copy source:/path dest:/path is what I'll likely end up using. In any case, the resulting solution still works.

                            Still need to tweak OSX a bit as it seems to kill the SSH session after some time for sporadic reasons - still testing but I may end up having to add ServerAliveInterval 120 to /etc/ssh/ssh_config to resolve that issue.

                            1 Reply Last reply Reply Quote 0
                            • 1 / 1
                            • First post
                              Last post