Cardiff has the Artes Mundi prize every two years and I make sure I visit. This guy won most recently.
http://www.artesmundi.org/en/artists/john-akomfrahHis piece was original, beautiful, poignant, meaningful and richly deserving of the prize. This is contemporary art but I understand that the proposed museum would be for both modern and contemporary art.
The Cardiff Centre for Visual Arts flopped in 2000 but that was down to the hefty entrance fee. I willingly paid it and enjoyed my visit but charging is not a good idea unless the gallery has an incredible collection or is in a wealthy, very touristy city.
The National Gallery has good collections of modern and contemporary art with pieces by Welsh artists. I think a museum of Welsh Art in Cardiff would be a good idea. There have always been permanent galleries/displays of "Welsh art through the ages" at the Cathays Park museum but a dedicated building somewhere nearby would be great, freeing up the rest of the museum for its collections of other art/artefacts.
I think that given that Cardiff has the National Museum/Gallery plus the National History Museum that siting a new gallery in Merthyr, Newport or Swansea should be considered. They are all very easy to reach from Cardiff and would give a huge boost to any of those places. Given the size of the collections held by National Museum Wales, maybe a disused building in each of our large towns and cities could be converted and pieces could tour, like they do with the Tate's collections?