Preheat oven to 350°F (175°C) and grease an 8x8-inch baking dish.
Soak raisins in hot water for 10 minutes, then drain.
Toast cubed gluten-free bread on a baking sheet for 5-10 minutes until lightly golden.
Whisk together eggs, brown sugar, granulated sugar, vanilla extract, cinnamon, and nutmeg in a medium bowl.
Melt butter and stir it with milk into the egg mixture until combined.
Layer half the toasted bread in the prepared dish, sprinkle with half the raisins, and repeat the layers.
Pour the egg mixture evenly over the bread and raisins, gently stirring to ensure all pieces are coated.
Bake for 30 minutes, then cover with foil and bake for an additional 20 minutes, or until a knife inserted into the center comes out clean and the edges are golden brown.