The problem that begins to emerge is two-fold. First off, if you have backups going on at the same time you are trying to do restores, there is contention for resources. Second, if you have a large amount of deduplicated data, the index will be too big to be kept in memory. So, some of the index is cached off to disk. Swapping the index out from memory to disk and back again can dramatically increase backup time.
Check out the entire Data Deduplication FAQ.