Preheat over to 350°F. Line an 8x8 inch square pan with parchment paper. Set aside.
To toast the coconut, pour coconut onto a sheet pan in an even layer. Place the baking sheet in the oven for about 8 minutes, stirring a couple of times, until the coconut is golden brown. Remove it from the oven and let it cool. Leave the oven on.
Put the graham crackers into a food processor and pulse until the crackers become fine crumbs. Add the toasted coconut, sugar and butter and continue pulsing until the mixture is wet (about 15 seconds). Press the mixture into the bottom of the prepared pan creating an even layer. Bake in the preheated oven for 10 minutes. Let cool completely. Leave the oven on.
In the bowl of an electric mixer add the egg yolks and lime zest. Beat on high for 5 minutes using the whisk attachment. Mixture will be very thick. Reduce the mixer speed to medium and add in the sweetened condensed milk in a slow steady stream. Increase the speed to high and mix for another 3 minutes. Reduce the mixer speed to low and add in the key lime juice, mix until the juice is combined.
Spread the filling evenly over the crust. Bake in the preheated oven for 10 minutes, rotating after 5 minutes. Cool completely, then refrigerate for at least 4 hours before cutting and serving. Garnish with whipped cream and a slice of lime if desired.