post

Quick vi tip: Show Hidden Characters

I do a lot of development in linux/unix. In fact, that’s where all my development is, but I spend a fair amount of time in the “CLI” or command line interface. I use putty and ssh in to edit files and look at logs and just generally do what I do.

Today I ran into a problem with a file and I couldn’t figure out what was wrong. I kept getting an error and the spacing was off so I started suspecting a hidden character. vi, the command line editor I use in linux can show you hidden characters. Just type:

:set list

And bingo, hidden characters are revealed.

Turning them back off is just as easy:

:set nolist

Write that down, it comes in handy.

Updated 2/25/10: Thanks to Stephen in the comments, I’ve updated the above commands. I had the : in the wrong place!

About Chrispian

My name is Chrispian and I like to make things. I'm a web developer and have been working in the computer / internet field for almost 22 years now. I also draw, write, take pictures and love figuring stuff out.

Comments

  1. Thanks for this info on vi. I didn’t want to drag out the books and figure out which option it was again! I really needed this for file conversion between Mac OS and Windows OS (the infamous carriage return versus line feed issue). Thanks again!

  2. You don’t need to post my response. I think you meant “:set list” and “:set nolist” instead of “set: list”. I got an error when I typed “set: list” and “:set: list”. Thought yuo might want to know so you can fix your post and save others the headache of figuring it out.

    This post was very helpful never the less. Thanks!

  3. Stephen, so right! Totally typo’d that. Updating it now.

  4. Once you find the characters you want to remove you can use:
    :%s///g to remove everything globally. The way to type those pesky special characters in vi (ie ^M) is to use ctrl-v then ctrl-m You then substitute the m for whatever letter you need.

    Hope that helps!

  5. Sorry the posting left out some of my characters. A sample of the above technique would be :%s/^M//g

    Sorry for the mixup!

  6. :%s/.$//g

    will take care of you dos file carry over ^M
    cheers\r

  7. Is there a way to show the hidden \t characters?

    Thanks,

  8. Also, if you like me would like a key-binding for this, you can use the :set list! version. Notice the “!” at the end. This will toggle list on and off.

  9. This is a very useful command. Thanks for sharing this.
    We were having troubles running a service using a config file as the config file has hidden characters. I was able to find out the hidden characters in the file using this command and it resolved out issue.
    Thanks very much!

  10. Josh Lee says:

    I found out today that :set list in vim doesn’t actually always show dos characters, only sometimes. To make 100% sure you need to run “cat” on the file with the two flags “v” and “T”. So if your file is named foo.bar, you’d type in: “cat -vT foo.bar”

  11. so much easier then getting up and going to my bookshelf to break out the o’reillys book. thanks

  12. Thank you for the Vi tip. I have been pulling my hair out over some CSS that is not rendering in the browser. Here is the CSS:

    .quiz-button-go:hover{
    color:#ddffff;
    background:#28e07e;
    }

    Simple right?! The problem is the color is not registering, but the background is. But, when I delete the spaces between the newline and “color”. it works perfectly.

    The CSS in question is CSS that I have cut and pasted from my browser into Vim.

    When I use :set list, what I get is

    44.quiz-button-go:hover{$
    45    $
    46 ^Icolor:#ddffff;$
    47 ^Ibackground:#28e07e;$
    48 }$

    I’m assuming ^I is tab and $ must be carriage return? Looks fine, right?

    Do you know of any other way to view any special hidden chars?

    I repeat, if I delete those few spaces, it works perfectly. So I know it’s not a CSS programming thing. This is totally bizarre.

    I’ve run into this kind of problem maybe twice in the last 4 months. It’s a real PITA because the text is indeed correct and hunting down this kind of bug is tough…

  13. Bradley Rentfrow says:

    This is what I found to removed the pesky ^M’s that mess up your script:
    :e ++ff=dos
    :setlocal ff=unix
    :w

    When you run cat -vT or run the script the ^M’s should be gone.
    Basically it saves the file back into the UNIX format and scripts the DOS carriage returns from the file.

  14. Thanks for the post. I now just have to decipher what I am seeing. Hmmm running :set list and then seeing some ^I or $ means what?

  15. Deniz Gezmis says:

    If you happen to run Slackware Linux the following command should take care of those DOS related characters smoothly.

    fromdos output_file.txt

  16. Michael Monteith says:

    On some systems the vi -b option works great. Here is a good note on the subject
    http://www.devdaily.com/blog/post/linux-unix/vi-editor-binary-mode-remove-control-m-characters

  17. Thanks for the post, it definitely help to save the reference, since I only use it like once in a while. :-)

  18. Terry Mackie says:

    I was using diff and by eye balling the code, I couldn’t figure out why 2 lines were different. I used set list and saw the 2 lines each had a space at the end. I removed the spaces and diff was sucessful.

  19. Hopping Bunny says:

    Helped me fix a strange problem that cropped up after I added some comments to a shell script !

    Go figure.

  20. Of course, if it is just the pesky ^M from a windows file that is the issue, there is always the good old “dos2unix” converston tool:
    dos2unix -o a.xtx b.txt

    That will automatically strip out the DOS line feed (carriage return code) stuff. HTH.

Add to the Conversation