ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Search text file for pattern

    Scheduled Pinned Locked Moved IT Discussion
    grep
    17 Posts 5 Posters 1.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      manxam @1337
      last edited by

      @Pete-S : grep can use regular expressions to search using the pcre syntax.
      If you could provide a sample of a piece of text that you want matched we may be able to help.
      Otherwise, use regexr.com and it'll help you build the expression.

      JaredBuschJ 1 Reply Last reply Reply Quote 2
      • JaredBuschJ
        JaredBusch
        last edited by

        https://www.digitalocean.com/community/tutorials/using-grep-regular-expressions-to-search-for-text-patterns-in-linux#regular-expressions

        1 Reply Last reply Reply Quote 2
        • JaredBuschJ
          JaredBusch @manxam
          last edited by

          @manxam said in Search text file for pattern:

          Otherwise, use regexr.com and it'll help you build the expression.

          nice tool. will have to remember it.

          1 Reply Last reply Reply Quote 0
          • 1
            1337
            last edited by

            Alright, I have the search expression down and regexr.com was a great interactive tool.
            /[<\.:;"]([A-Z]*T4[A-Z,0-9]+)[>\.:;"]+/g

            However, how do I get grep to deliver the match (capturing group) and not the complete lines?

            JaredBuschJ 1 Reply Last reply Reply Quote 0
            • JaredBuschJ
              JaredBusch @1337
              last edited by

              @Pete-S said in Search text file for pattern:

              Alright, I have the search expression down and regexr.com was a great interactive tool.
              /[<\.:;"]([A-Z]*T4[A-Z,0-9]+)[>\.:;"]+/g

              However, how do I get grep to deliver the match (capturing group) and not the complete lines?

              Don’t use grep? I have no idea what tool you actually want here.

              1 1 Reply Last reply Reply Quote 0
              • 1
                1337 @JaredBusch
                last edited by

                @JaredBusch said in Search text file for pattern:

                @Pete-S said in Search text file for pattern:

                However, how do I get grep to deliver the match (capturing group) and not the complete lines?

                Don’t use grep? I have no idea what tool you actually want here.

                Me neither. I just want to search a file and get a list of what matches.

                travisdh1T 1 Reply Last reply Reply Quote 0
                • travisdh1T
                  travisdh1 @1337
                  last edited by

                  @Pete-S said in Search text file for pattern:

                  @JaredBusch said in Search text file for pattern:

                  @Pete-S said in Search text file for pattern:

                  However, how do I get grep to deliver the match (capturing group) and not the complete lines?

                  Don’t use grep? I have no idea what tool you actually want here.

                  Me neither. I just want to search a file and get a list of what matches.

                  grep should be able to do that natively now. We used to cat filename | grep

                  JaredBuschJ 1 Reply Last reply Reply Quote 0
                  • JaredBuschJ
                    JaredBusch @travisdh1
                    last edited by

                    @travisdh1 said in Search text file for pattern:

                    @Pete-S said in Search text file for pattern:

                    @JaredBusch said in Search text file for pattern:

                    @Pete-S said in Search text file for pattern:

                    However, how do I get grep to deliver the match (capturing group) and not the complete lines?

                    Don’t use grep? I have no idea what tool you actually want here.

                    Me neither. I just want to search a file and get a list of what matches.

                    grep should be able to do that natively now. We used to cat filename | grep

                    Grep returns the whole line where the string was found. He does not want that. He only wants to results of the regex.

                    1 1 Reply Last reply Reply Quote 1
                    • 1
                      1337 @JaredBusch
                      last edited by 1337

                      @JaredBusch said in Search text file for pattern:

                      @travisdh1 said in Search text file for pattern:

                      @Pete-S said in Search text file for pattern:

                      @JaredBusch said in Search text file for pattern:

                      @Pete-S said in Search text file for pattern:

                      However, how do I get grep to deliver the match (capturing group) and not the complete lines?

                      Don’t use grep? I have no idea what tool you actually want here.

                      Me neither. I just want to search a file and get a list of what matches.

                      grep should be able to do that natively now. We used to cat filename | grep

                      Grep returns the whole line where the string was found. He does not want that. He only wants to results of the regex.

                      OK, I think I have it.

                      -o, --only-matching
                      Print only the matched (non-empty) parts of a matching line, with each such part on a separate output line.

                      So grep pattern file -o should do the trick and it seems like it from my first test.

                      1 1 Reply Last reply Reply Quote 2
                      • 1
                        1337 @1337
                        last edited by 1337

                        @Pete-S said in Search text file for pattern:

                        @JaredBusch said in Search text file for pattern:

                        @travisdh1 said in Search text file for pattern:

                        @Pete-S said in Search text file for pattern:

                        @JaredBusch said in Search text file for pattern:

                        @Pete-S said in Search text file for pattern:

                        However, how do I get grep to deliver the match (capturing group) and not the complete lines?

                        Don’t use grep? I have no idea what tool you actually want here.

                        Me neither. I just want to search a file and get a list of what matches.

                        grep should be able to do that natively now. We used to cat filename | grep

                        Grep returns the whole line where the string was found. He does not want that. He only wants to results of the regex.

                        OK, I think I have it.

                        -o, --only-matching
                        Print only the matched (non-empty) parts of a matching line, with each such part on a separate output line.

                        So grep pattern file -o should do the trick and it seems like it from my first test.

                        OK, so grep can match an expression and deliver the match with -o option but not part of the match, aka capturing group. Grep doesn't have the full capabilities of regular expressions, only parts of it.

                        But pcregrep does.

                        So in my case I put the search expression in pattern.txt because I had some odd characters in it. So pcregrep delivered the goods with:
                        pcregrep -o1 -f pattern.txt file_to_search

                        So the take home message is that pcregrep is like grep on steroids.
                        Most of the options and usage is the same as grep so it's a drop in replacement in most cases.

                        PS. I tried with debian and on debian you need the pcregrep package, which you can install with apt install pcregrep. I don't know if it's installed by default on fedora/centos.

                        scottalanmillerS 1 Reply Last reply Reply Quote 0
                        • scottalanmillerS
                          scottalanmiller @1337
                          last edited by

                          @Pete-S said in Search text file for pattern:

                          Grep doesn't have the full capabilities of regular expressions, only parts of it.

                          Not sure what this means. RE is a kind of thing, there is no such thing as a full set of RE capabilities. Some systems are more powerful than others. But there is no standard RE. It's not like relational databases which were defined before the first one was created as to what RDBs were and what was required to be fully relational. RE is just a concept of pattern matching, nothing more. Anything that does pattern matching is doing RE.

                          1 1 Reply Last reply Reply Quote 1
                          • 1
                            1337 @scottalanmiller
                            last edited by

                            @scottalanmiller said in Search text file for pattern:

                            @Pete-S said in Search text file for pattern:

                            Grep doesn't have the full capabilities of regular expressions, only parts of it.

                            Not sure what this means. RE is a kind of thing, there is no such thing as a full set of RE capabilities. Some systems are more powerful than others. But there is no standard RE. It's not like relational databases which were defined before the first one was created as to what RDBs were and what was required to be fully relational. RE is just a concept of pattern matching, nothing more. Anything that does pattern matching is doing RE.

                            OK, you are right.

                            The regular expressions used in javascript, php and others are perl compatible. PCRE stands for Perl Compatible Regular Expressions. pcregrep supports these but grep does not.

                            According to wikipedia:

                            PCRE's syntax is much more powerful and flexible than either of the POSIX regular expression flavors and than that of many other regular-expression libraries.

                            scottalanmillerS 1 Reply Last reply Reply Quote 0
                            • scottalanmillerS
                              scottalanmiller @1337
                              last edited by

                              @Pete-S said in Search text file for pattern:

                              @scottalanmiller said in Search text file for pattern:

                              @Pete-S said in Search text file for pattern:

                              Grep doesn't have the full capabilities of regular expressions, only parts of it.

                              Not sure what this means. RE is a kind of thing, there is no such thing as a full set of RE capabilities. Some systems are more powerful than others. But there is no standard RE. It's not like relational databases which were defined before the first one was created as to what RDBs were and what was required to be fully relational. RE is just a concept of pattern matching, nothing more. Anything that does pattern matching is doing RE.

                              OK, you are right.

                              The regular expressions used in javascript, php and others are perl compatible. PCRE stands for Perl Compatible Regular Expressions. pcregrep supports these but grep does not.

                              Yes, PERL made a non-standard RE system long after GREP and other mainline RE tools were commonplace. PERL is the weird one here, and later systems copied them. But PERL went their own route long after GREP and others were standard.

                              1 1 Reply Last reply Reply Quote 0
                              • 1
                                1337 @scottalanmiller
                                last edited by

                                @scottalanmiller said in Search text file for pattern:

                                @Pete-S said in Search text file for pattern:

                                @scottalanmiller said in Search text file for pattern:

                                @Pete-S said in Search text file for pattern:

                                Grep doesn't have the full capabilities of regular expressions, only parts of it.

                                Not sure what this means. RE is a kind of thing, there is no such thing as a full set of RE capabilities. Some systems are more powerful than others. But there is no standard RE. It's not like relational databases which were defined before the first one was created as to what RDBs were and what was required to be fully relational. RE is just a concept of pattern matching, nothing more. Anything that does pattern matching is doing RE.

                                OK, you are right.

                                The regular expressions used in javascript, php and others are perl compatible. PCRE stands for Perl Compatible Regular Expressions. pcregrep supports these but grep does not.

                                Yes, PERL made a non-standard RE system long after GREP and other mainline RE tools were commonplace. PERL is the weird one here, and later systems copied them. But PERL went their own route long after GREP and others were standard.

                                You know, before this thread I never took notice that there was an older standard in place and that some of the tools used it. I just though everything regex was the same, maybe with some minor implementation specific changes.

                                The reason for that is of course that every time I had to do some more advanced regex, at least the last 10 years or so, it has been in mod_rewrite, php, javascript or something like that. And as it turns out, they are all using pcre.

                                scottalanmillerS 1 Reply Last reply Reply Quote 0
                                • scottalanmillerS
                                  scottalanmiller @1337
                                  last edited by

                                  @Pete-S said in Search text file for pattern:

                                  @scottalanmiller said in Search text file for pattern:

                                  @Pete-S said in Search text file for pattern:

                                  @scottalanmiller said in Search text file for pattern:

                                  @Pete-S said in Search text file for pattern:

                                  Grep doesn't have the full capabilities of regular expressions, only parts of it.

                                  Not sure what this means. RE is a kind of thing, there is no such thing as a full set of RE capabilities. Some systems are more powerful than others. But there is no standard RE. It's not like relational databases which were defined before the first one was created as to what RDBs were and what was required to be fully relational. RE is just a concept of pattern matching, nothing more. Anything that does pattern matching is doing RE.

                                  OK, you are right.

                                  The regular expressions used in javascript, php and others are perl compatible. PCRE stands for Perl Compatible Regular Expressions. pcregrep supports these but grep does not.

                                  Yes, PERL made a non-standard RE system long after GREP and other mainline RE tools were commonplace. PERL is the weird one here, and later systems copied them. But PERL went their own route long after GREP and others were standard.

                                  You know, before this thread I never took notice that there was an older standard in place and that some of the tools used it. I just though everything regex was the same, maybe with some minor implementation specific changes.

                                  The reason for that is of course that every time I had to do some more advanced regex, at least the last 10 years or so, it has been in mod_rewrite, php, javascript or something like that. And as it turns out, they are all using pcre.

                                  Yeah, I come from the pre-PERL GREP era and so PERL was this annoying upstart that broke all of our de facto standards 😞

                                  1 Reply Last reply Reply Quote 1
                                  • M
                                    manxam
                                    last edited by

                                    I should have noted that grep likely wouldn't like complicated RE patterns. Sed, however, supports the full gamut and is installed by default on pretty much every linux variant; not sure if pcregrep is.

                                    It seems everyone likes the link I posted for the RE builder so I'll paste two others that I use:
                                    https://regex101.com/
                                    http://leaverou.github.io/regexplained/

                                    Cheers!

                                    1 Reply Last reply Reply Quote 2
                                    • 1 / 1
                                    • First post
                                      Last post