ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Downloading full Website offline

    Scheduled Pinned Locked Moved IT Discussion
    wgetwebsitedownload
    41 Posts 9 Posters 2.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • scottalanmillerS
      scottalanmiller @travisdh1
      last edited by

      @travisdh1 said in Downloading full Website offline:

      @black3dynamite said in Downloading full Website offline:

      @dbeato @scottalanmiller

      https://github.com/mariomaric/website-size

      That is literally saving the website to a temp directory. Why not just do it once instead of twice?

      Right, which is what I had originally said you could do. And doing so to a temp directory is STILL downloading the whole thing - the very thing you are trying to avoid.

      1 Reply Last reply Reply Quote 1
      • DustinB3403D
        DustinB3403
        last edited by

        @dbeato are you specifying a user account for this to run again?

        scottalanmillerS dbeatoD 2 Replies Last reply Reply Quote 0
        • scottalanmillerS
          scottalanmiller @DustinB3403
          last edited by

          @DustinB3403 said in Downloading full Website offline:

          @dbeato are you specifying a user account for this to run again?

          No, he'd have to pass a cookie to do that.

          DustinB3403D JaredBuschJ 2 Replies Last reply Reply Quote 1
          • dbeatoD
            dbeato @DustinB3403
            last edited by

            @DustinB3403 said in Downloading full Website offline:

            @dbeato are you specifying a user account for this to run again?

            I am not.

            1 Reply Last reply Reply Quote 0
            • DustinB3403D
              DustinB3403 @scottalanmiller
              last edited by

              @scottalanmiller okay. . .

              So I know exactly why @dbeato is going through this process. The followup is then, how do you sort out the things you don't want from the website and retain those for later?

              dbeatoD 1 Reply Last reply Reply Quote 0
              • dbeatoD
                dbeato @DustinB3403
                last edited by

                @DustinB3403 said in Downloading full Website offline:

                @scottalanmiller okay. . .

                So I know exactly why @dbeato is going through this process. The followup is then, how do you sort out the things you don't want from the website and retain those for later?

                That is what I would do slowly in my spare time 🙂

                DustinB3403D 1 Reply Last reply Reply Quote 0
                • JaredBuschJ
                  JaredBusch @scottalanmiller
                  last edited by

                  @scottalanmiller said in Downloading full Website offline:

                  @DustinB3403 said in Downloading full Website offline:

                  @dbeato are you specifying a user account for this to run again?

                  No, he'd have to pass a cookie to do that.

                  48a5ebae-e0bc-45b5-81d5-a549493376fd-image.png

                  1 Reply Last reply Reply Quote 1
                  • DustinB3403D
                    DustinB3403 @dbeato
                    last edited by

                    @dbeato said in Downloading full Website offline:

                    @DustinB3403 said in Downloading full Website offline:

                    @scottalanmiller okay. . .

                    So I know exactly why @dbeato is going through this process. The followup is then, how do you sort out the things you don't want from the website and retain those for later?

                    That is what I would do slowly in my spare time 🙂

                    Ugh. . .

                    1 Reply Last reply Reply Quote 0
                    • black3dynamiteB
                      black3dynamite @scottalanmiller
                      last edited by

                      @scottalanmiller said in Downloading full Website offline:

                      @black3dynamite said in Downloading full Website offline:

                      @dbeato @scottalanmiller

                      https://github.com/mariomaric/website-size

                      But does it really not download everything? Websites don't report on the size directly AFAIK.

                      Using --spider, does not download the pages.
                      Using --no-directories, tells wget to create empty directories.

                      scottalanmillerS 1 Reply Last reply Reply Quote 0
                      • scottalanmillerS
                        scottalanmiller @black3dynamite
                        last edited by

                        @black3dynamite said in Downloading full Website offline:

                        @scottalanmiller said in Downloading full Website offline:

                        @black3dynamite said in Downloading full Website offline:

                        @dbeato @scottalanmiller

                        https://github.com/mariomaric/website-size

                        But does it really not download everything? Websites don't report on the size directly AFAIK.

                        Using --spider, does not download the pages.
                        Using --no-directories, tells wget to create empty directories.

                        So it gets it all from "Content Length"? Interesting. Guess that would work.

                        1 Reply Last reply Reply Quote 0
                        • 1
                        • 2
                        • 3
                        • 3 / 3
                        • First post
                          Last post