this is a time consuming exercise but worth it in the long run