Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

I'm writing a small method to replace some text in a file. The only argument I need is the new text, as it is always the same file and text to be replaced.

I'm having a problem using the os.system() call, when I try to use the argument of the method

If I use a string like below, everything runs ok:

stringId = "GRRRRRRRRR"
cmd="sed '1,$s/MANAGER_ID=[0-9]*/MANAGER_ID=" + stringId + "/g' path/file.old > path/file.new"
os.system(cmd)
Now, if i try to give a string as a parameter like below, the command is not executed.
I do a print to see if the command is correct, and it is. I can even execute it with success if I copy / paste to my shell
import os
def updateExportConfigId(id):
stringId = "%s" % id
cmd= "sed '1,$s/MANAGER_ID=[0-9]*/MANAGER_ID=" + stringId + "/g' path/file.old > path/file.new"
print "command is " + cmd
os.system(cmd)
Does anyone knows what is wrong?
Thanks
                Just a quick note. The line     stringId = "%s" % id is completely useless. "%s" % id is exactly the same as str(id), which if id is a string (which it should be in your example) of course in itself is useless.
– Lennart Regebro
                Jul 1, 2009 at 13:50
                What version of Python are you running?  Other than following the suggestions to change the id variable name and drop the stringID line (and also setting the paths to legitimate values on my system), I got your code to run fine on Python 2.4.3, even without the str(cmd).  One other thing, the way you have this set up, you'll be changing the all the occurences of MANAGER_ID=some_id. Maybe this is what you want if you only have 1 MANAGER_ID in your file, but if you more than 1, you'll be setting all of them to the same value.
– PTBNL
                Jul 1, 2009 at 15:09
def updateExportConfigId(m_id, source='path/file.old', 
                             destination='path/file.new'):
    if isinstance(m_id, unicode):
        m_id = m_id.encode('utf-8')
    cmd= [
          "sed",
          ",$s/MANAGER_ID=[0-9]*/MANAGER_ID=%s/g" % m_id,  
          source,
    subprocess.call(cmd, stdout=open(destination, 'w'))
with this code you can pass the manager id, it can have spaces, quote chars, etc. The file names can also be passed to the function, and can also contain spaces and some other special chars. That's because your shell is not unnecessarly invoked, so one less process is started on your OS, and you don't have to worry on escaping special shell characters.
Another option: Don't launch sed. Use python's re module. 
import re
def updateExportConfigID(m_id, source, destination):
    if isinstance(m_id, unicode):
        m_id = m_id.encode('utf-8')
    for line in source:
        new_line = re.sub(r'MANAGER_ID=\d*', 
                          r'MANAGER_ID=' + re.escape(m_id), 
                          line)
        destination.write(new_line)
and call it like this:
updateExportConfigID('GRRRR', open('path/file.old'), open('path/file.new', 'w'))
No new processes needed.
def updateExportConfigId(id):
    stringId = "%s" % id
    cmd= "sed '1,$s/MANAGER_ID=[0-9]*/MANAGER_ID=" + stringId + "/g' test.dat > test.new"
    print "command is " + cmd
    os.system(cmd)
updateExportConfigId("adsf")
Also do not use reserved words (id) as variables.
                I already had tried that and the result is the same. Using the print repr(cmd), I can see there is a difference between these 2 cases: cmd = 'mv test.dat test.' + stringId output for print repr(cmd) is: u'mv test.dat test.new' where I used the "new" string as argument But if  cmd="sed '1,$s/MANAGER_ID=[0-9]*/MANAGER_ID=" + stringId + "/g' test.dat > test.new" output for print repr(cmd) is: u"sed '1,$s/MANAGER_ID=[0-9]*/MANAGER_ID=adsf/g' test.dat > test.new"  I tried all string combinations using " ", ' ' , %s, ''+'', " " + " ", ...
– Swon
                Jul 1, 2009 at 13:29
                Try actually using the same example when you explain the difference between the two cases. The only difference I can see between the two cases is that you use completely different commands. That's not very helpful.
– Lennart Regebro
                Jul 1, 2009 at 13:43
                For correctness, id is not a reserved word. It is a built-in function. The point of not using them still applies.
– nosklo
                Jul 2, 2009 at 2:19
What is wrong is that there is some difference. Yeah, I know that's not helpful, but you need to figure out the difference.
Try running this:
import os
def updateExportConfigId(id):
    stringId = "%s" % id
    cmd1 = "sed '1,$s/MANAGER_ID=[0-9]*/MANAGER_ID=" + stringId + "/g' path/file.old > path/file.new"
    stringId = "GRRRRRRRRR"
    cmd2 = "sed '1,$s/MANAGER_ID=[0-9]*/MANAGER_ID=" + stringId + "/g' path/file.old > path/file.new"
    print "cmd1:" , cmd1
    print "cmd2:" , cmd2
    print cmd1 == cmd2
updateExportConfigId("GRRRRRRRRR")
The code should print:
sed '1,$s/MANAGER_ID=[0-9]*/MANAGER_ID=GRRRRRRRRR/g' path/file.old > path/file.new
sed '1,$s/MANAGER_ID=[0-9]*/MANAGER_ID=GRRRRRRRRR/g' path/file.old > path/file.new
Thereby showing that they are exactly the same. If the last line is "False" then they are not the same, and you should be able to see the difference.
                Well, I really getting out hair..  I tested and the strings are the same.  The difference appears only with print repr()  print repr(command1): u"sed '1,$s/MANAGER_ID=[0-9]*/MANAGER_ID=GRRRR/g' file.old > file.new"  print repr(cmd2): "sed '1,$s/MANAGER_ID=[0-9]*/MANAGER_ID=GRRRR/g' file.old > file.new"  In this case, cmd2 is using hardcoded string and I'm able to run cmd2 with os.system() call.  Thanks
– Swon
                Jul 1, 2009 at 14:18
                Aha, so the difference is that in the first case it's unicode, and the other it's a string. And for some reason calling os.system() with unicode doesn't work then.  That must mean that the variable id getting passed in to the method is unicode.
– Lennart Regebro
                Jul 1, 2009 at 14:41
So from previous answers we now know that id is a Unicode string, which makes cmd1 a Unicode string, which os.system() is converting to a byte string for execution in the default encoding.
a) I suggest using subprocess rather than os.system()
b) I suggest not using the name of a built-in function as a variable (id).
c) I suggest explicitly encoding the string to a byte string before executing:
if isinstance(cmd,unicode):
    cmd = cmd.encode("UTF-8")
d) For Lennart Regebro's suggestion add:
assert type(cmd1) == type(cmd2)
after
print cmd1 == cmd2
Finally, I found a way to run the os.system(cmd)!
Simple trick, to "clean" the cmd string:
os.system(str(cmd))
Now, I'm able to build the cmd with all arguments I need and at the end I just "clean" it with str() call before run it with os.system() call.
Thanks a lot for your answers!
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.