Some rare or protected species of fish in the game can't be cooked. For example, the mythical fish that only appears once in a blue moon.
Well, usually the fish that are marked as special or ornamental within the game's mechanics can't be cooked. Like those super colorful ones that are more for display.
In Story of Seasons, certain fish that have unique ecological roles or are tied to specific in-game events can't be cooked. Maybe they are part of a quest or have some other non-culinary significance.