Preheat the oven to 425 degrees F. Place the halibut in a single layer in a parchment paper-lined baking dish. Drizzle with the orange juice; season with salt and pepper. Bake the halibut for 12 to 15 minutes, or until the fish is cooked through.
While the fish bakes, heat the oil in a large skillet set over medium-high heat. Add the asparagus and stir-fry 2 minutes. Add the ginger and garlic and cook 30 seconds more. Add the teriyaki/cornstarch mixture and chili sauce, if using, bring to a simmer, and simmer 1 minute.
When the fish is cooked, set a fillet on each of four plates. Spoon an equal amount of the asparagus and its sauce over the fish.