fix deduping before input filtering