ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    What Are You Doing Right Now

    Scheduled Pinned Locked Moved Water Closet
    time waster
    88.9k Posts 285 Posters 42.8m Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • scottalanmillerS
      scottalanmiller @thwr
      last edited by

      @thwr said in What Are You Doing Right Now:

      @scottalanmiller said in What Are You Doing Right Now:

      @thwr said in What Are You Doing Right Now:

      How much I love zsync. Wrote a client for our gaming community last year. Unlike rsync, there's no direct server connection beyond HTTP required.

      Just synced 18.6 GB worth of updated files, took me three minutes and 20 seconds with 100 MBit downstream. Mostly disk-I/O.

      http://zsync.moria.org.uk/

      Awesome for syncing files because it only transmits changed blocks. It's much more efficient than expected, saving up to 95% traffic on our bills.

      Already thought about building a .NET lib. There are only Java libs available. It's not that I dislike Java. No. I hate it.

      It's rsync under the hood.

      thwrT 1 Reply Last reply Reply Quote 0
      • thwrT
        thwr @scottalanmiller
        last edited by

        @scottalanmiller said in What Are You Doing Right Now:

        @thwr said in What Are You Doing Right Now:

        @scottalanmiller said in What Are You Doing Right Now:

        @thwr said in What Are You Doing Right Now:

        How much I love zsync. Wrote a client for our gaming community last year. Unlike rsync, there's no direct server connection beyond HTTP required.

        Just synced 18.6 GB worth of updated files, took me three minutes and 20 seconds with 100 MBit downstream. Mostly disk-I/O.

        http://zsync.moria.org.uk/

        Awesome for syncing files because it only transmits changed blocks. It's much more efficient than expected, saving up to 95% traffic on our bills.

        Already thought about building a .NET lib. There are only Java libs available. It's not that I dislike Java. No. I hate it.

        It's rsync under the hood.

        Same algorithm, yes.

        scottalanmillerS 1 Reply Last reply Reply Quote 0
        • scottalanmillerS
          scottalanmiller @thwr
          last edited by

          @thwr said in What Are You Doing Right Now:

          @scottalanmiller said in What Are You Doing Right Now:

          @thwr said in What Are You Doing Right Now:

          @scottalanmiller said in What Are You Doing Right Now:

          @thwr said in What Are You Doing Right Now:

          How much I love zsync. Wrote a client for our gaming community last year. Unlike rsync, there's no direct server connection beyond HTTP required.

          Just synced 18.6 GB worth of updated files, took me three minutes and 20 seconds with 100 MBit downstream. Mostly disk-I/O.

          http://zsync.moria.org.uk/

          Awesome for syncing files because it only transmits changed blocks. It's much more efficient than expected, saving up to 95% traffic on our bills.

          Already thought about building a .NET lib. There are only Java libs available. It's not that I dislike Java. No. I hate it.

          It's rsync under the hood.

          Same algorithm, yes.

          That's what rsync is 😉

          thwrT 1 Reply Last reply Reply Quote 0
          • thwrT
            thwr @scottalanmiller
            last edited by

            @scottalanmiller said in What Are You Doing Right Now:

            @thwr said in What Are You Doing Right Now:

            @scottalanmiller said in What Are You Doing Right Now:

            @thwr said in What Are You Doing Right Now:

            @scottalanmiller said in What Are You Doing Right Now:

            @thwr said in What Are You Doing Right Now:

            How much I love zsync. Wrote a client for our gaming community last year. Unlike rsync, there's no direct server connection beyond HTTP required.

            Just synced 18.6 GB worth of updated files, took me three minutes and 20 seconds with 100 MBit downstream. Mostly disk-I/O.

            http://zsync.moria.org.uk/

            Awesome for syncing files because it only transmits changed blocks. It's much more efficient than expected, saving up to 95% traffic on our bills.

            Already thought about building a .NET lib. There are only Java libs available. It's not that I dislike Java. No. I hate it.

            It's rsync under the hood.

            Same algorithm, yes.

            That's what rsync is 😉

            There's a saying here... "Counting peas?" 😉

            1 Reply Last reply Reply Quote 0
            • black3dynamiteB
              black3dynamite
              last edited by

              I did it again. I ended up using ctrl+shift+v in Windows.

              1 Reply Last reply Reply Quote 0
              • coliverC
                coliver @thwr
                last edited by

                @thwr said in What Are You Doing Right Now:

                @reid-cooper said in What Are You Doing Right Now:

                Have not heard of zsync, that seems pretty useful.

                The basic idea is that you upload your files to your webserver and generate a data block map using zsyncmake. This must be repeated every time you update a file. That's all you need on the server side.

                On the client, you just invoke zsync and point it to the zsync map files (which contains the block map and the relative or absolute URL to the real file). It then compares your local version and starts to sync changed blocks.

                We are syncing nearly 60GB of mostly binary files (game mods) for more than 300 highly active users. Generating the map files and a lot of other stuff (like JSON listings and hashes) takes us about 10 minutes on an average SSD.

                Are they authenticating somehow? Or does it just point to the directory?

                thwrT 1 Reply Last reply Reply Quote 0
                • thwrT
                  thwr @coliver
                  last edited by thwr

                  @coliver said in What Are You Doing Right Now:

                  @thwr said in What Are You Doing Right Now:

                  @reid-cooper said in What Are You Doing Right Now:

                  Have not heard of zsync, that seems pretty useful.

                  The basic idea is that you upload your files to your webserver and generate a data block map using zsyncmake. This must be repeated every time you update a file. That's all you need on the server side.

                  On the client, you just invoke zsync and point it to the zsync map files (which contains the block map and the relative or absolute URL to the real file). It then compares your local version and starts to sync changed blocks.

                  We are syncing nearly 60GB of mostly binary files (game mods) for more than 300 highly active users. Generating the map files and a lot of other stuff (like JSON listings and hashes) takes us about 10 minutes on an average SSD.

                  Are they authenticating somehow? Or does it just point to the directory?

                  It's plain HTTP, so whatever your webserver can do.

                  There's only one downside: No HTTPS support. I do not like the idea, but that's another story. You can work around by many means: SSH port forwarding or VPN tunnels, for example.

                  thwrT 1 Reply Last reply Reply Quote 0
                  • thwrT
                    thwr @thwr
                    last edited by thwr

                    @thwr said in What Are You Doing Right Now:

                    @coliver said in What Are You Doing Right Now:

                    @thwr said in What Are You Doing Right Now:

                    @reid-cooper said in What Are You Doing Right Now:

                    Have not heard of zsync, that seems pretty useful.

                    The basic idea is that you upload your files to your webserver and generate a data block map using zsyncmake. This must be repeated every time you update a file. That's all you need on the server side.

                    On the client, you just invoke zsync and point it to the zsync map files (which contains the block map and the relative or absolute URL to the real file). It then compares your local version and starts to sync changed blocks.

                    We are syncing nearly 60GB of mostly binary files (game mods) for more than 300 highly active users. Generating the map files and a lot of other stuff (like JSON listings and hashes) takes us about 10 minutes on an average SSD.

                    Are they authenticating somehow? Or does it just point to the directory?

                    It's plain HTTP, so whatever your webserver can do.

                    There's only one downside: No HTTPS support. I do not like the idea, but that's another story. You can work around by many means: SSH port forwarding or VPN tunnels, for example.

                    I did a small workaround, as our content (mods) is not confidential. A script puts rolling credentials into the .htaccess-file of the webservers directory, keeping only the newest X entries. Invoked via cron. A serverside webservice provides the credentials to authenticated clients. So even if someone sniffes users and passwords (both random), he could only use them for a short amount of time.

                    Not ideal, but like I said, it's not required to be secure. Just a small barrier.

                    1 Reply Last reply Reply Quote 0
                    • EddieJenningsE
                      EddieJennings
                      last edited by

                      Headache healed enough to allow me to play string with my cat before I head to sleep. Day = salvaged 😄

                      1 Reply Last reply Reply Quote 0
                      • JaredBuschJ
                        JaredBusch
                        last edited by

                        Piss off Windows 7 updates temp files...
                        0_1517546632715_386ffd79-b2e6-41c3-9096-bdf4edcd535c-image.png

                        dbeatoD EddieJenningsE 2 Replies Last reply Reply Quote 1
                        • dbeatoD
                          dbeato @JaredBusch
                          last edited by

                          @jaredbusch said in What Are You Doing Right Now:

                          Piss off Windows 7 updates temp files...
                          0_1517546632715_386ffd79-b2e6-41c3-9096-bdf4edcd535c-image.png

                          Such a pain...

                          1 Reply Last reply Reply Quote 0
                          • scottalanmillerS
                            scottalanmiller
                            last edited by

                            Just wrapped up our family game night.

                            1 Reply Last reply Reply Quote 0
                            • hobbit666H
                              hobbit666
                              last edited by

                              Just ordered Sausage and Chips for my Lunch.......Hurry up dinner time!!!!

                              J 1 Reply Last reply Reply Quote 0
                              • J
                                JackCPickup @hobbit666
                                last edited by

                                @hobbit666 Thanks, you've decided my lunch for me. To the chippy!

                                1 Reply Last reply Reply Quote 1
                                • EddieJenningsE
                                  EddieJennings @JaredBusch
                                  last edited by

                                  @jaredbusch said in What Are You Doing Right Now:

                                  Piss off Windows 7 updates temp files...

                                  Bleh!

                                  1 Reply Last reply Reply Quote 0
                                  • dbeatoD
                                    dbeato
                                    last edited by

                                    Day off today!!

                                    DustinB3403D 1 Reply Last reply Reply Quote 1
                                    • DustinB3403D
                                      DustinB3403 @dbeato
                                      last edited by

                                      @dbeato said in What Are You Doing Right Now:

                                      Day off today!!

                                      Lucky, it's getting bad outside, we're supposed to get 3-6" of snow today.

                                      1 Reply Last reply Reply Quote 0
                                      • Minion QueenM
                                        Minion Queen
                                        last edited by

                                        0_1517579891334_40521e00-abe5-0133-b34c-0e438b3b98d1.jpg

                                        1 Reply Last reply Reply Quote 5
                                        • scottalanmillerS
                                          scottalanmiller
                                          last edited by

                                          Good morning.

                                          1 Reply Last reply Reply Quote 0
                                          • EddieJenningsE
                                            EddieJennings
                                            last edited by

                                            Funtimes documentation. 😄

                                            1 Reply Last reply Reply Quote 0
                                            • 1
                                            • 2
                                            • 2735
                                            • 2736
                                            • 2737
                                            • 2738
                                            • 2739
                                            • 4443
                                            • 4444
                                            • 2737 / 4444
                                            • First post
                                              Last post