ConcurrentDictionary<TKey,TValue> is a new type in the .NET Framework 4, living in the System.Collections.Concurrent namespace.  As noted in the MSDN documentation, ConcurrentDictionary “represents a thread-safe collection of key-value pairs that can be accessed by multiple threads concurrently.” 

While ConcurrentDictionary implements IDictionary<TKey, TValue> just as does Dictionary<TKey,TValue>, it also has several unique mechanisms for adding / updating key/value pairs in the dictionary, mechanisms specific to its concurrent focus.  Here is a summary of when to use which mechanism.

• If you want to…
• Add a new item to the dictionary only if the key doesn’t currently exist in the dictionary…
• Use TryAdd.  TryAdd accepts the key and the value, and adds the pair to the dictionary if the key doesn’t currently exist in the dictionary.  The method returns whether or not the new pair was added.
• public bool TryAdd(TKey key, TValue value)
• Update the value for an existing key in the dictionary, but only if the existing value for that key is equal to a specific value…
• Use TryUpdate.  TryUpdate accepts the key and the new value to set that key to, but it also accepts the value that the key in the dictionary must currently contain in order for the update to succeed.  You can think of TryUpdate like Interlocked.CompareExchange but for an element in the dictionary.
• public bool TryUpdate(TKey key, TValue newValue, TValue comparisonValue)
• Store a key/value pair into the dictionary unconditionally, overwriting any value for that key if the key already exists…
• Use the indexer’s setter, e.g. dictionary[key] = newValue.
• public TValue this[TKey key] { get; set; }
• Add a key/value pair to the dictionary if the key doesn’t exist in the dictionary, or if the key does exist, update the value for the key based on the key’s existing value…
• Use AddOrUpdate.  AddOrUpdate has two overloads:
• One overload accepts the key as well as two delegates.  The first delegate is used if the key doesn’t exist in the dictionary; it accepts the key and returns the value that should be added for the key.  The second delegate is used if the key does exist: it accepts both the key and the current value for the key, and it returns the new value that should be set for the key.
• public TValue AddOrUpdate(TKey key, Func<TKey, TValue> addValueFactory, Func<TKey, TValue, TValue> updateValueFactory)
• The other overload accepts the key, an add value, and the update delegate.  This is exactly the same as the former overload, except that if the key isn’t in the dictionary, the provided value is added for the key, rather than invoking a delegate to get the necessary value
• public TValue AddOrUpdate(TKey key, TValue addValue, Func<TKey, TValue, TValue> updateValueFactory)
• Get the value for a key in the dictionary, adding the value to the dictionary (and returning it) if the key doesn’t exist…
• Use GetOrAdd.  You can think of this as lazy-initialization for a key/value pair in the dictionary, getting the value if it’s there, or adding it and then getting it if it’s not.  As with AddOrUpdate, GetOrAdd provides two overloads: one takes the value to be added if the key doesn’t exist, and the other takes a delegate that will generate the value if the key doesn’t exist.
• public TValue GetOrAdd(TKey key, TValue value)
• public TValue GetOrAdd(TKey key, Func<TKey, TValue> valueFactory)

All of these operations are atomic and are thread-safe with regards to all other operations on the ConcurrentDictionary.  The only caveat to the atomicity of each operation is for those which accept a delegate, namely AddOrUpdate and GetOrAdd.  For modifications / writes to the dictionary, ConcurrentDictionary employs fine-grained locking to ensure thread-safety (reads on the dictionary are performed in a lock-free manner); however, these delegates are invoked outside of the locks in order to avoid the myriad of problems that can arise from executing unknown code under a lock.  As such, the code executed by these delegates is not subject to the atomicity of the operation.