Why do we care if skilled immigrants are leaving the US?

Should we really be worried if some Indians and Chinese decide to head back to the countries of their birth? Some number crunching by Vivek Wadhwa points to the fact that we absolutely should be concerned and the administration should work on increasing the annual cap count of skilled worker visas available.

Because immigrants are critical to the country's long-term economic health. Despite the fact that they constitute only 12% of the U.S. population, immigrants have started 52% of Silicon Valley's technology companies and contributed to more than 25% of our global patents. They make up 24% of the U.S. science and engineering workforce holding bachelor's degrees and 47% of science and engineering workers who have PhDs.
..The average age was 30 for Indian returnees, and 33 for Chinese. They were highly educated, with degrees in management, technology, or science. Fifty-one percent of the Chinese held master's degrees and 41% had PhDs. Sixty-six percent of the Indians held a master's and 12.1% had PhDs.
..Eighty-seven percent of Chinese and 62% of Indians said they had better opportunities for longer-term professional growth in their home countries than in the U.S. Additionally, nearly half were considering launching businesses and said entrepreneurial opportunities were better in their home countries than in the U.S.