Industry guru Dave Taylor answers free tech support questions about a wide variety of business and technical topics, including blogging, Google AdSense, MySpace, Sony PSP, Apple iPod, Mp3 players, management, Linux, SEO, Mac OS X, Facebook, Twitter, LinkedIn and Microsoft Windows.

How do I flatten a directory structure in Unix?

Dave, how can I flatten a directory structure? I've got a directory "foo", with many more sub-directories and files within. I want to take all the files from all levels beneath foo and put them in foo itself.

I've tried this: find . *.mp3 -print0 | xargs -0 mv . but I get an error on the mv command that I can't figure out. Help!


Dave's Answer:

You're definitely on the right path here. To extract all files in a subdirectory the find command is the correct program, and using the Mac-specific extension "-print0" coupled with the xargs "-0" lets you handle those annoying filenames with spaces in them.

There are two problems with what you have here, though: first off, the syntax of mv is essentially move a to b, but you're ending up supplying the arguments in the wrong order, because xargs appends the filenames to what you specify, essentially creating "mv . folder/file1 folder/file2", etc. You can see where that's going to confuse the command!

The second problem is a bit more subtle: as the program proceeds, I wouldn't bet that the '.' is going to be interpreted properly and I would strongly suggest that instead you specify the directory name of your target anyway.

Unfortunately, the xargs man page is pretty darn confusing and doesn't help much. What you want to do is use the -J flag, specify a pattern (or single character), then embed that in the subsequent command. Everwhere the pattern appears, it'll be replaced by the stream of filenames produced by the find command.

This'll make more sense with an example, so let's jump in!

$ pwd
/Users/taylor/Desktop/foo
$ find . -type f -name "*mp3" -print0 | \
   xargs -0 -J% mv % /Users/taylor/Desktop/foo
$

That will do what you seek. When you're ready to delete all the now-empty directories, don't forget you can use find again, just specify -type d to match directories but not files.

Good luck!



Help others find this article at Del.icio.us, Digg, Netscape, Reddit, and Simpy.

Subscribe!

Never miss another useful Q&A article again! Subscribe to AskDaveTaylor with Google Reader.

Comments

How about one step further?

I'd like to make a cron job that backs up only the protected iTunes music files in my iTunes directory complete with directory structure. I've tried various cp commands and never can seem to figure it out.

How would you go about doing that?

Posted by: Tom Boucher at January 31, 2005 9:56 PM

Thanks for responding!

OK, that looks great, but -print0 is implemented on my Suse 9.1 box just fine (not a Mac!). Also, -J isn't implemented in Suse 9.1, so I'm still stuck.

I'd also add `pwd` to the end so as to not have to hard code the destination so it can be put in a shell script.

Posted by: Phil at January 31, 2005 11:17 PM

Linux also has -print0 in find and -0 in xargs. -J is not in Linux but I guess that --replace can be used instead. The one qustion I have though is why not use

find . -type f -name "*.mp3" -exec cp {} \
/Users/taylor/Desktop/foo

instead of find and xargs?

Posted by: Avi at February 1, 2005 3:02 AM

Good point, Avi, but you ironically demonstrate why I avoid the curly brace notation in find by your mistake: In fact, you need to escape the curly braces AND you need to have an escaped semicolon too for it to work properly, as in:

find . -type f -name "*.mp3" -exec cp \{\} /Users/taylor/Desktop/foo \;

A good solution, though, if you're willing to fiddle around and get the backslashes right. :-)

Posted by: Dave Taylor at February 1, 2005 3:09 AM

"I'd like to make a cron job that backs up only the protected iTunes music files in my iTunes directory complete with directory structure."

I don't know what "protected" means in this context, but I'll say that if you want to back up an entire directory structure, you can typically use either cp -R or cpio (use 'man cpio' to see how this useful program works).

Posted by: Dave Taylor at February 1, 2005 3:52 AM

Here's the final, working command for linux:

find . -type f -name "*mp3" -print0 | xargs -0 --replace=% mv % `pwd`

Posted by: Phil at February 1, 2005 2:12 PM

Great. Thanks for coming back and completing the loop. I knew that the GNU version of xargs had to have some solution to this.

Posted by: Dave Taylor at February 1, 2005 2:25 PM

"Protected" in the context "protected iTunes music files" refers to files of type .m4p, which are AAC files (ordinarily .m4a) that are "protected" with Apple's Playfair DRM wrapper.

Posted by: Ken at February 1, 2005 6:06 PM

Okay, Ken, this is new to me, even though I'm a big fan of my two iPods and iTunes on all my Macs. Are you asking if there's a way to unprotect the files and/or defeat the Playfair DRM?

Posted by: Dave Taylor at February 1, 2005 6:55 PM

Um ... how about using the Finder?

In the Finder, File:Find...

Search in: "Specific Places"

Click on "Add" button and choose the foo folder

Search for items whose name ends with .mp3

The select all the results and drag them to foo.

Posted by: David Rouse at February 12, 2005 5:37 PM

David, I don't think that'll work because he's trying to pull files from lots of different subdirectories at the same time, and even in "list" view the Finder has a tendency to want to move the folders that files are in when you do large multi-directory grabs...

Posted by: Dave Taylor at February 12, 2005 6:24 PM

The 'find | xargs' examples should run a lot faster then a 'find ... --exec' operation as you'd have to create one process for each file as opposed to xargs which will make as few processes as possible based on the length of the data on the stdin.

On Solaris I usually will do something like 'find ... | sed -e "s/.*/'&'/g" | xargs ...'. This will wrap each line in single quotes.

Keep up the good tips.

Posted by: Anacreo at March 2, 2005 8:14 PM

-print0 is a GNU/BSD extension !

Posted by: monsieur at May 17, 2006 5:18 AM

I have a lot to say, but ...
Starbucks coffee cup I have a lot to say, and questions of my own for that matter, but most of all I'd like to say thank you for all your efforts on this Web site by buying you a chai!

I do have a comment, now that you mention it!









Remember personal info?


Please note that I will never send you any unsolicited commercial email. Ever.

While I'm at it, please note that by submitting a question or comment you're agreeing to my terms of service, which are: you relinquish any subsequent rights of ownership to your material by submitting it on this site.









Search
Find just the answers you seek from among our 1700+ free tech support articles by using our Lijit search engine.


Help!





Subscribe to
Ask Dave Taylor!

Add to Google Reader
Add to My Yahoo!
Subscribe in NewsGator Online

RDF   XML

Free Updates!
Sign up and get free weekly updates and special offers on books, seminars, workshops and more.


Recent Entries
Join the List!
Join my author info mailing list, where you'll learn about my upcoming books, speaking gigs, and more!


Book Links
© 2002 - 2008 by Dave Taylor. All Rights Reserved.

Note: This web site is for the purpose of disseminating information for educational purposes, free of charge, for the benefit of all visitors. We take great care to provide quality information. However, we do not guarantee, and accept no legal liability whatsoever arising from or connected to, the accuracy, reliability, currency or completeness of any material contained on this web site or on any linked site.

[whiteboard marker tray]