Excluding files or directories while deleting in Linux

When you use a graphical UI, removing a selected group of files or directories is easy! You just click a few, then delete.

On the command line (in Linux, for example), this isn't so easy.

Today I needed to clear out all Wordpress themes in a blog, except for the one currently in use.

How did I do it?


cd /var/www/mywordpresssite/wp-content/themes/
rm -rf `ls | grep -v themetokeep/ | grep -vF \.`


The above command removes all files and directories EXCEPT themetokeep, and those with dots in them. The latter is VERY important. If you include '..' in the rm-rf command, it will attempt to delete everything up one directory as well. In this case, that would have been the wp-content/ directory - yep, not a good idea!

Let's break that command down a bit:

ls

- the simplest way to list files. It just lists the names (no file size, dates, etc) of all files and directories in the current directory. Some systems are configured to hide the . and .. directories when ls is used with no parameters. However, some are not - which is why we need to be careful by adding that last grep.

|

- the pipe directs the output from one command to the input of another. In this case, we are sending the output of the ls command to the grep command, instead of printing the ls command's output to the shell.

grep -v themetokeep/

- grep - a great program that applies regular expressions to any input you give it, and outputs the result. It is essentially a filter.
- We are using the -v parameter to invert the match. This means we want to output all files and directories that do NOT have the name themetokeep/
If we didn't include -v and simply types grep themetokeep/ then we'd get an output list which only includes that directory.

grep -vF \.

- So now we have the output of the ls command, filtered to exclude the directory themetokeep/, and now we are filtering it one more time to remove the . and .. directories.

This is really important because we don't want to delete the current directory (wp-content/themes/) and we DEFINITELY don't want to delete the parent directory (..) - as this would remove all our plugins, cache, uploads, etc.. Please don't make that mistake as I once did! :)

- The -F parameter - to be honest, I am not exactly sure why we need this, except that if we don't use it, we won't be able to match the directories with dots in them. So use it.

- We put the slash before the . so that it is treated as a literal dot, and not the regular expression match character, which means 'any character'

rm -rf `...`

- rm - the delete command. We need -r to allow us to delete directories (and all their subdirectories). We need -f to suppress the 'Are you sure?' prompts. In other words, we're telling rm to delete everything we tell it - no questions asked.
- We put the filtered file list within the backticks ``, so that the file list becomes the instruction to rm.


Hope that helps you  - any questions - leave them in the comments!

Encrypting ASP.Net Configuration Files

Required Tools

%WinDir%\Microsoft.NET\Framework\<versionNumber>\aspnet_regiis.exe
IIS 7 (Instructions vary slightly for Windows 2000 and 2003) – see link in last section.

(Please excuse the formatting - haven't had time to make it pretty)

Overview

The purpose of this document is to explain how to encrypt sections of ASP.Net configuration files. This is necessary to add security around things like database and email account passwords.

While the config files are protected from public web users, in the event that a server was hacked, the config file contents could be accessed. Encrypted sensitive data in config files means that this data will remain safer than if it was stored in plain text.

We want to be able to:

• Easily work with config files locally
• Easily encrypt them on Production servers
• Be able to do the encryption as part of the deployment

There are two approaches to encrypting configuration files:

1. Using the supplied tools to encrypt the files on each server, using the server's own private key.
2. Using a common private key on all servers, and encrypting the config files locally before deploying.

We will go with approach 2) because it reduces the server management overhead, and speeds up deployment.

Step 1: Generating the Private Keys

This only needs to be done once.

1. On any Windows machine (preferably a Server version), open a Command Prompt window with Admin privileges (in the Start menu, hold down Ctrl and Shift, and click on Accessories->Command Prompt)

Ideally, we need to use a Server operating system because this tool does not work 100% on Workstation OS’s such as Vista. You can still try this on XP or Vista though, as long as you have IIS - it will probably still work.

2. First, we create an RSA key container.

In the command prompt, type:

%SYSTEMROOT%\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis.exe -pc RandomtechKeyContainer –exp

The name “RandomtechKeyContainer” may be substituted for anything however, this name will be used in the configuration later – so make it something that makes sense.

The –exp parameter means that the keys will be exportable.

Output:
Creating RSA Key container...
Succeeded! 

3. Second, we create the private and public key and save them to a file.

Type:

%SYSTEMROOT%\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis -px RandomtechKeyContainer thekeys.xml –pri 

The –pri switch means the public AND private keys are exported.

Output:

Exporting RSA Keys to file...
Succeeded!

4. Now we have the private and public keys saved in a file. Download this file to your PC and save it somewhere. The contents of the file should look something like:


(Keys mangled for my safety!)

Step 2: Setting up Your Development PC for Use with Keys

Now we have created the keys, we need to import them into our PC’s key container store so we can easily use them to encrypt configuration files.

1. Open a command prompt (the easiest one to use is the Visual Studio 2008 Command Prompt, found in the Start Menu) and type:

%SYSTEMROOT%\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis -pi RandomtechKeyContainer YOUR_PCs_PATH_TO\thekeys.xml
 
(change YOUR_PCs_PATH_TO to where you saved the keys)

Output:

Importing RSA Keys from file..
Succeeded!

Step 3: Encrypting the Config Files

Now we can use the above key to encrypt our files.

1. Add the following section to your application’s web.config file:


<configProtectedData>
<providers>
<add name="RandomtechCustomEncryptionProvider"
type="System.Configuration.RsaProtectedConfigurationProvider, System.Configuration, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"
keyContainerName="RandomtechKeyContainer"
useMachineContainer="true" />
</providers>
</configProtectedData>

This can be added anywhere in the configuration file, directly within the configuration element (so not within system.web, etc.)

Make sure the keyContainerName is exactly the same as the key container name you used above, while generating keys.

Setting useMachineContainer to true means that the application will search for the decryption keys in the machine key store. This is important for use on the web server, otherwise it will search for the key in a specific user’s Application Data directory. Using this approach, means the key will be retrieved from the “All Users” directory.

2. Now we are ready to actually encrypt some configuration file sections. We’ll do the ConnectionStrings first, but it’s also recommended to encrypt your AppSettings, and any other section with sensitive data.
In a command prompt with Admin privileges, type:

%SYSTEMROOT%\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis -pef “connectionStrings” "FILE_PATH_TO_YOUR_WEB_APP" –prov “RandomtechCustomEncryptionProvider” 


The string after –prov must match the provider name we used above in the config section declaration.

The above command will look for all configuration files in the location FILE_PATH_TO_YOUR_WEB_APP. This location is a physical directory location on your PC. For example, c:\dev\project2

Note: the path MUST NOT END WITH A SLASH, otherwise the command will fail.


3. If you would like to specify a path to a web application on your local IIS server, then you need to drop the “f” from –pef, and supply a virtual path with the –app parameter. The command will look like this:

aspnet_regiis -pe “connectionStrings” –app “IIS_VIRTUAL_PATH” –prov “RandomtechCustomEncryptionProvider”
 
e.g.

aspnet_regiis -pe “connectionStrings” –app “/MyWebApp” –prov “RandomtechCustomEncryptionProvider” 


Note: if you are using the Visual Studio Web Server, use the –pef version of the command above.

4. Your connectionStrings will have been transformed into the encrypted version. It looks like this: 


<connectionStrings configProtectionProvider="RandomtechCustomEncryptionProvider">
<EncryptedData Type="http://www.w3.org/2001/04/xmlenc#Element"
xmlns="http://www.w3.org/2001/04/xmlenc#">
<EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc" />
<KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
<EncryptedKey xmlns="http://www.w3.org/2001/04/xmlenc#">
<EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5" />
<KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
<KeyName>Rsa Key</KeyName>
</KeyInfo>
<CipherData>
<CipherValue>kjytgleldK+…</CipherValue>
</CipherData>
</EncryptedKey>
</KeyInfo>
<CipherData>
<CipherValue>cdznrkA11o1uatlHtmxFwQ2HXWEE/…</CipherValue>
</CipherData>
</EncryptedData>
</connectionStrings>
(Encrypted data removed of course! But you get the idea…)

5. Now test that decryption works:

e.g.
aspnet_regiis -pdf “connectionStrings” “c:\dev\project2”
OR
aspnet_regiis -pd “connectionStrings” –app “/MyWebApp”

Step 4: Incorporating Encrypted Configuration Into Deployment

We’ve done everything we need to do to encrypt/decrypt keys on our local machines. Now we need to automate it so we don’t even have to think about it. The following example uses NAnt for automation.
You can include the following NAnt snippet in your build files. Note we use NAnt argument elements for each of the exec/program parameter so we can use double quotes in the command line.


<property name="dotnet" value="c:/windows/Microsoft.NET/Framework/v2.0.50727" overwrite="true" />

<property name="build.target" value="..\..\DeployOutput\webapp.deploy" overwrite="false" />

<exec basedir="."
program="${dotnet}/aspnet_compiler.exe"
commandline="-nologo -u -d -c -v ${build.webappname} ${build.target} "
workingdir="."
failonerror="true" />

<!-- Encrypt the connection strings before deploying -->
<exec program="${dotnet}/aspnet_regiis.exe"
basedir="${build.target}"
verbose="true"
failonerror="true">
<arg value="-pef" />
<arg value='"connectionStrings"' />
<arg value="${path::get-full-path(build.target)}" />
<arg value='-prov "RandomtechCustomEncryptionProvider"' />
</exec>

If you need help with NAnt, get in touch via the comments.

Step 5: Distributing Keys to Web Servers

In order for our web servers to be able to decrypt the encrypted configuration, they need the private key.
To install it on each web server, the same method is followed as above for the local PC.

1. Upload the key to the server

2. Create the container and import the keys, type:

aspnet_regiis -pi RandomtechKeyContainer mykeys.xml

Output:

Importing RSA Keys from file..
Succeeded!

3. The last step is to grant access to the key container to the user your web application runs as. In IIS 7, it is the Application Pool identity, type:

%SYSTEMROOT%\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis -pa "RandomtechKeyContainer" "IIS APPPOOL\AppName"


Output:

Adding ACL for access to the RSA Key container...
Succeeded!

AppName is your application pool’s name.


4. Bonus Tip!

To find out the user your application pool runs as, add an ASPX file to your web root, with the following contents:

<%@ Page Language="VB" %>
<%
Response.Write(System.Security.Principal.WindowsIdentity.GetCurrent().Name)
%>

Open the file and copy this name into the command line above.
You will need to do this for each application pool that needs access to the key container.

5. Now delete the key file from the server. It is no longer needed, and for security reasons, it should not be left on the server.

6. The server is now ready to decrypt your configuration. Deploy your application and test an operation where the encrypted configuration section is needed. The application should function as normal!

Useful Points and References

1. Sometimes the aspnet_regiis command will fail and just print out the usage instructions. If this happens, try surrounding every parameter with double quotes “”. It seems to require them sometimes, even when the parameter has no spaces in it.
2. http://msdn.microsoft.com/en-us/library/ms998283.aspx
3. For info on Windows 2000/2003: http://jtoee.com/2008/02/encrypting-webconfig/
4. In Windows, RSA keys are stored here: \Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA\MachineKeys