Tsuki ga Michibiku Isekai Douchuu

Makoto Misumi is summoned to another world under an agreement made between its goddess and his parents, supposedly to become a hero. The goddess takes one look at him, declares him ugly, strips him of the title, and throws him into the wasteland at the edge of civilization. Her rejection comes with one strange compensation: Makoto can understand nearly every language except that of humans. Far from powerless, he discovers that the new world's conditions have awakened extraordinary physical and magical abilities in him. His journey soon attracts dragons, orcs, dwarves, and other nonhuman peoples, forcing Makoto to build a place for himself among those the goddess's favored society treats as outsiders.