Wordpress cpt tax_query brake attachment size and srcset

Question

I'm using this tax_query for a custom post type and custom taxonomy but it looks like it breaks the attachment size. My attachments load in full size, and every time I remove my tax_query from my post query, the attachment loads as it should be.

 $query_args = array(
    'post_type'      => 'knowledge',
    'posts_per_page' => 99,
    'tax_query'      => array(
        array(
            'taxonomy' => 'knowledge_category',
            'field'    => 'term_id',
            'terms'    => 316,
        ),
    ),
 );

Any idea how to solve this ?