Hi folks, I'm banging my head against how to properly pass a list of arguments in a variable through to a command in bash where the list of arguments itself has quoted values in it. Here's a test case script: $ cat odd-quoting.sh #!/bin/bash a='--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign' echo cpan2rpm $a If I run that script with dash on Ubuntu it looks fine: $ sh -x odd-quoting.sh + a=--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign + echo cpan2rpm --debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign [...] But if I run it with bash on either Ubuntu or my actual target, RHEL6, it gets it wrong: $ bash -x odd-quoting.sh + a='--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign' + echo cpan2rpm --debug '--make-maker=--make-maker="INSTALLDIRS=vendor' INSTALLMAN1DIR=none 'INSTALLMAN3DIR=none"' --no-sign [...] I get the same problem if I run bash from its /bin/sh symlink on RHEL6. Any ideas what I'm missing here ? cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
You will want to make $a an array, e.g.: a=(arg1 arg2) a+=(arg3) Then unpack properly: "${a[@]}" Here's a pretty good guide: http://mywiki.wooledge.org/BashGuide/Arrays Cheers, Dave On May 2, 2012 4:49 PM, "Chris Samuel" <chris@csamuel.org <mailto:chris@csamuel.org>> wrote: Hi folks, I'm banging my head against how to properly pass a list of arguments in a variable through to a command in bash where the list of arguments itself has quoted values in it. Here's a test case script: $ cat odd-quoting.sh #!/bin/bash a='--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign' echo cpan2rpm $a If I run that script with dash on Ubuntu it looks fine: $ sh -x odd-quoting.sh + a=--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign + echo cpan2rpm --debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign [...] But if I run it with bash on either Ubuntu or my actual target, RHEL6, it gets it wrong: $ bash -x odd-quoting.sh + a='--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign' + echo cpan2rpm --debug '--make-maker=--make-maker="INSTALLDIRS=vendor' INSTALLMAN1DIR=none 'INSTALLMAN3DIR=none"' --no-sign [...] I get the same problem if I run bash from its /bin/sh symlink on RHEL6. Any ideas what I'm missing here ? cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC _______________________________________________ luv-main mailing list luv-main@luv.asn.au <mailto:luv-main@luv.asn.au> http://lists.luv.asn.au/listinfo/luv-main
On 2 May 2012 17:56, David Schoen <dave@lyte.id.au> wrote:
You will want to make $a an array, e.g.: a=(arg1 arg2) a+=(arg3)
As soon as you start using array's, the script is no longer POSIX sh compliant, you have to use bash or zsh or something. Personally I think as soon as you start needing to use array's in shell scripting, that is a good sign that maybe something like Python might be more appropriate... -- Brian May <brian@microcomaustralia.com.au>
On 03/05/12 09:19, Brian May wrote:
As soon as you start using array's, the script is no longer POSIX sh compliant, you have to use bash or zsh or something.
That's OK as it's a shell script one person (i.e. me), or possibly one other may ever run (once the RPMs are built then we should never need this again, but it's good to be able to automate just in case). It's only ever going to run on RHEL6 PPC64 anyway. :-)
Personally I think as soon as you start needing to use array's in shell scripting, that is a good sign that maybe something like Python might be more appropriate...
If I wanted to be told how to write my programs I'd use Ada.. ;-) cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
On 02.05.12 16:49, Chris Samuel wrote:
I'm banging my head against how to properly pass a list of arguments in a variable through to a command in bash where the list of arguments itself has quoted values in it.
Maybe I'm missing something, but if the enclosing single quotes need to be omitted, as in the dash example, then why not just omit them?: $ a=--debug --make-maker=--make-maker="INSTALLDIRS=vendor \ INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign $ echo cpan2rpm $a cpan2rpm --debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign If, on the other hand, it's the weird placement of the single quotes which offends, then instead of trying to carry them over, I'd just place them where and when needed: $ echo cpan2rpm \'$a\' cpan2rpm '--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign' Is that getting anywhere near understanding the problem? Erik
On 2 May 2012 18:33, Erik Christiansen <dvalin@internode.on.net> wrote:
Maybe I'm missing something, but if the enclosing single quotes need to be omitted, as in the dash example, then why not just omit them?:
$ a=--debug --make-maker=--make-maker="INSTALLDIRS=vendor \ INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign
err... Did you try that? brian@aquitard:~$ a=--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign --make-maker=--make-maker=INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none: command not found Because the value is not quoted, the shell is parsing it as: ENV=VALUE CMD ARGS which is not what you want. -- Brian May <brian@microcomaustralia.com.au>
On 03.05.12 09:27, Brian May wrote:
On 2 May 2012 18:33, Erik Christiansen <dvalin@internode.on.net> wrote:
Maybe I'm missing something, but if the enclosing single quotes need to be omitted, as in the dash example, then why not just omit them?:
$ a=--debug --make-maker=--make-maker="INSTALLDIRS=vendor \ INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign
err... Did you try that?
Err ... Did you read my post? Did you not observe that it was tried in the very next line of my post, in a line duplicating Chris'? Even though the parameters were only following an "echo", I gave consideration to the quoting differences in the two examples cited by Chris, since that was the nature of the stated problem. In the second part, I offered a way to place enclosing quotes without intervening shell behaviour being able to impose portability issues. A shorter quoting example would have been preferable, but after repeated scanning of the long-winded example, the discernible difference was placement of single quotes. If the proposed quoting fix (in the second part) does not work for Chris, then another example, showing what it does do there, would be a basis for further discussion. On the other hand, if he has fixed it, then we're all happy. Erik -- "What is wanted is not the will to believe, but the will to find out, which is the exact opposite." - Bertrand Russell, _Sceptical_Essays_, 1928
On 2 May 2012 16:49, Chris Samuel <chris@csamuel.org> wrote: <...>
But if I run it with bash on either Ubuntu or my actual target, RHEL6, it gets it wrong:
Oddly enough, the quoting works as you expected on my install of RHEL6 bash 4.1.2(1)-release and on my install of Debian bash 4.2.5(1)-release Perhaps there's something I'm missing? -- Joel Shea <jwshea@gmail.com>
On Wed, 02 May 2012 16:49:07 +1000, Chris Samuel <chris@csamuel.org> wrote:
#!/bin/bash a='--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign' echo cpan2rpm $a
You can also solve it with eval, but it is a tad ugly: a="--debug --make-maker=--make-maker=\"INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none\" --no-sign" eval "echo cpan2rpm $a" I reckon that Dave's suggestion of an array is much cleaner: a=(--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign) echo cpan2rpm "${a[@]}" naturally, neither of them work in dash! Glenn -- sks-keyservers.net 0xb1e82ec9228ac090
On 2012-05-02 16:49, Chris Samuel wrote:
Hi folks,
I'm banging my head against how to properly pass a list of arguments in a variable through to a command in bash where the list of arguments itself has quoted values in it.
Here's a test case script:
$ cat odd-quoting.sh #!/bin/bash
a='--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign'
echo cpan2rpm $a
If I run that script with dash on Ubuntu it looks fine:
$ sh -x odd-quoting.sh + a=--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign + echo cpan2rpm --debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign [...]
But if I run it with bash on either Ubuntu or my actual target, RHEL6, it gets it wrong:
$ bash -x odd-quoting.sh + a='--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign' + echo cpan2rpm --debug '--make-maker=--make-maker="INSTALLDIRS=vendor' INSTALLMAN1DIR=none 'INSTALLMAN3DIR=none"' --no-sign [...]
I get the same problem if I run bash from its /bin/sh symlink on RHEL6.
Any ideas what I'm missing here ?
* Which versions of bash and dash are you using on each host? * 'echo' will theoretically expand the quotes, and therefore you won't see how the arguments actually get passed to cpan2rpm. * Regarding one of the arguments themselves: --make-maker=--make-maker=... surely that's a typo? * I'm not sure of your overall goal, but based on what I've seen, I think you're going about this the wrong way: cpan2rpm() { command cpan2rpm --debug \ --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none \ INSTALLMAN3DIR=none" --no-sign $@; } * If you actually need to do it using a variable, one thought process I followed briefly was this, but it didn't work as the double-quotes didn't get expanded. I mention it only because somebody else might take it and run with it: mattcen@toto:tmp$ read -r a <<EOF > --debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign > EOF mattcen@toto:tmp$ echo $a --debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign * Alternatively, for something really ugly[1]: mattcen@toto:tmp$ parse-args() { while [ "$#" -gt 0 ]; do echo "$1"; shift; done; } mattcen@toto:tmp$ a='--debug@--make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none"@--no-sign' mattcen@toto:tmp$ parse-args $a --debug@--make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none"@--no-sign mattcen@toto:tmp$ export OIFS="$IFS" mattcen@toto:tmp$ export IFS=@ mattcen@toto:tmp$ parse-args $a --debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign mattcen@toto:tmp$ export IFS="$OIFS" -- Regards, Matthew Cengia
Matthew Cengia wrote:
* I'm not sure of your overall goal, but based on what I've seen, I think you're going about this the wrong way:
cpan2rpm() { command cpan2rpm --debug \ --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none \ INSTALLMAN3DIR=none" --no-sign $@; }
That should be "$@" not $@.
From past experience with coworkers, that should probably be
obviously_not_foo () { command foo -x -y -z "$@"; } because they are prone to missing the function at the top of the script and wonder why foo is working further down, when half its arguments are missing. Otherwise, the aforementioned bash array approach is reasonable. xs=(foo bar baz # comment quux quux quuux) # another comment printf '==%s==\n' "${xs[@]}"
On 2012-05-02 21:25, Trent W. Buck wrote:
Matthew Cengia wrote:
* I'm not sure of your overall goal, but based on what I've seen, I think you're going about this the wrong way:
cpan2rpm() { command cpan2rpm --debug \ --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none \ INSTALLMAN3DIR=none" --no-sign $@; }
That should be "$@" not $@.
Ah yes, oops. Thanks for that.
From past experience with coworkers, that should probably be
obviously_not_foo () { command foo -x -y -z "$@"; }
because they are prone to missing the function at the top of the script and wonder why foo is working further down, when half its arguments are missing.
That too. -- Regards, Matthew Cengia
On Wed, May 02, 2012 at 09:14:17PM +1000, Matthew Cengia wrote:
* Which versions of bash and dash are you using on each host?
someone already mentioned that it worked as expected with RHEL bash 4.1.2(1)-release and Debian bash 4.2.5(1)-release. OTOH, it's pretty easy to read the wrong line in the ouput and read the echo output instead of the '+' trace output....i've done that twice so far this morning (perhaps due to ENOCOFFEE), and initially thought that the bash versions on sid and precise were working as expected, until i looked again and realised they weren't. I'm getting the same symptoms as Chris with debian sid's bash version 4.2.20(1)-release and the same on Ubuntu Precise with bash 4.2.24(1)-release, so maybe there's a bug with 4.2.20 (or thereabouts) and later. quoting in sh and bash has always been a PITA, but this seems just broken.
* 'echo' will theoretically expand the quotes, and therefore you won't see how the arguments actually get passed to cpan2rpm.
i've also tried it with /bin/echo rather than the builtin echo - same result, so it's not echo expanding the quotes, it's bash itself. hmmm. compare the difference between using echo and "cat <<__EOF__" #!/bin/bash a='--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign' echo cpan2rpm $a cat <<__EOF__ cpan2rpm $a __EOF__ $ bash -x /tmp/quote.sh + a='--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign' + echo cpan2rpm --debug '--make-maker=--make-maker="INSTALLDIRS=vendor' INSTALLMAN1DIR=none 'INSTALLMAN3DIR=none"' --no-sign cpan2rpm --debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign + cat cpan2rpm --debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign (i've added extra lines to make it more readable) that looks OK, so maybe running the cpan2rpm command with eval or in a sub-shell would work. craig -- craig sanders <cas@taz.net.au> BOFH excuse #351: PEBKAC (Problem Exists Between Keyboard And Chair)
Craig Sanders wrote:
OTOH, it's pretty easy to read the wrong line in the ouput and read the echo output instead of the '+' trace output....i've done that twice so far this morning (perhaps due to ENOCOFFEE), and initially thought that the bash versions on sid and precise were working as expected, until i looked again and realised they weren't. [...]
* 'echo' will theoretically expand the quotes, and therefore you won't see how the arguments actually get passed to cpan2rpm.
i've also tried it with /bin/echo rather than the builtin echo - same result, so it's not echo expanding the quotes, it's bash itself.
Use printf for this, not echo. bash4$ printf '==%s==\n' foo 'bar baz' "quux 'quuux quuuux'" ==foo== ==bar baz== ==quux 'quuux quuuux'==
On 03/05/12 09:16, Craig Sanders wrote:
someone already mentioned that it worked as expected with RHEL bash 4.1.2(1)-release and Debian bash 4.2.5(1)-release.
Well it wasn't working with this RHEL6 bash.. # rpm -q bash bash-4.1.2-8.el6.ppc64 All sorted using a function rather than mucking around with variables, so all is well with the world (OK, in my office) now. :-) cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
On 02/05/12 21:14, Matthew Cengia wrote:
* I'm not sure of your overall goal, but based on what I've seen, I think you're going about this the wrong way:
cpan2rpm() { command cpan2rpm --debug \ --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none \ INSTALLMAN3DIR=none" --no-sign $@; }
Yup, I think you're quite right, and with the addition of Trent's quoting it all works nicely again (and is much more elegant). Thanks everyone for your suggestions! All the best, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
On 2 May 2012 16:49, Chris Samuel <chris@csamuel.org> wrote:
If I run that script with dash on Ubuntu it looks fine:
It *looks* fine. Is it really fine though? I suspect not. In fact, I think the results are identical, just the debug information is different. Try this test script: --- cut --- #!/bin/sh -ex a='aaa bbb "ccc ddd"' cat $a --- cut --- On Ubuntu I 11.10 I get: --- cut --- + a=aaa bbb "ccc ddd" + cat aaa bbb "ccc ddd" cat: aaa: No such file or directory cat: bbb: No such file or directory cat: "ccc: No such file or directory cat: ddd": No such file or directory --- cut --- Not that the quotes inside the variable don't mean anything. Just another character. The is because the quotes are parsed on the command line *before* the variable is expanded. On RHEL6 I get: --- cut --- tsm ~ # ./test + a='aaa bbb "ccc ddd"' + cat aaa bbb '"ccc' 'ddd"' cat: aaa: No such file or directory cat: bbb: No such file or directory cat: "ccc: No such file or directory cat: ddd": No such file or directory --- cut --- For comparison, zsh is somewhat different: --- cut --- +./test:3> a='aaa bbb "ccc ddd"' +./test:4> cat 'aaa bbb "ccc ddd"' cat: aaa bbb "ccc ddd": No such file or directory --- cut --- Not that the debug output is different, however the results are exactly the same. Apologies if this has already been mentioned, I am not reading this thread in order. Also apologies for my hacked way of showing how the arguments are really split up :-) -- Brian May <brian@microcomaustralia.com.au>
Try using eval around the command: COMMAND='program --debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign' eval $COMMAND You need bash to re-evaluate those quotes so it understands it should be one argument. Jobst On Wed, May 02, 2012 at 04:49:07PM +1000, Chris Samuel (chris@csamuel.org) wrote:
Hi folks,
I'm banging my head against how to properly pass a list of arguments in a variable through to a command in bash where the list of arguments itself has quoted values in it.
Here's a test case script:
$ cat odd-quoting.sh #!/bin/bash
a='
echo cpan2rpm $a
If I run that script with dash on Ubuntu it looks fine:
$ sh -x odd-quoting.sh + a=--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign + echo cpan2rpm --debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign [...]
But if I run it with bash on either Ubuntu or my actual target, RHEL6, it gets it wrong:
$ bash -x odd-quoting.sh + a='--debug --make-maker=--make-maker="INSTALLDIRS=vendor INSTALLMAN1DIR=none INSTALLMAN3DIR=none" --no-sign' + echo cpan2rpm --debug '--make-maker=--make-maker="INSTALLDIRS=vendor' INSTALLMAN1DIR=none 'INSTALLMAN3DIR=none"' --no-sign [...]
I get the same problem if I run bash from its /bin/sh symlink on RHEL6.
Any ideas what I'm missing here ?
cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC _______________________________________________ luv-main mailing list luv-main@luv.asn.au http://lists.luv.asn.au/listinfo/luv-main
-- Time flies like the wind. Fruit flies like a banana. | |0| | Jobst Schmalenbach, jobst@barrett.com.au, General Manager | | |0| Barrett Consulting Group P/L & The Meditation Room P/L |0|0|0| +61 3 9532 7677, POBox 277, Caulfield South, 3162, Australia
participants (10)
-
Brian May -
Chris Samuel -
Craig Sanders -
David Schoen -
Erik Christiansen -
Glenn McIntosh -
Jobst Schmalenbach -
Joel W Shea -
Matthew Cengia -
Trent W. Buck