At the risk of sounding like an insufferable know-it-all... my understanding is this:
Higher Bit Depth --> More Dynamic Range
Higher Sample Rate --> Increased frequency range (upwards)
Many modern DAWs do their internal processing at 32-Bit, as the increased precision keeps things cleaner when doing a lot of edits/processing.
"Floating Point" processing is also related to better audio quality during processing... but even those two features don't seem to be critical/audible to a lot of engineers, since Pro Tools didn't have either feature for a long time, while most other DAWs had both for years, before PT followed suit. The "Pros" still continued to praise Pro Tools as if it was the only viable choice (LOL).
So, an increased bit depth absolutely makes sense, and will have a direct and audible impact on dynamic range (e.g. if you want to record an explosion and a whisper in the same take, better record at the highest bit-depth available).
Sample rate, on the other hand, is all about the frequency range that gets reproduced. I'm sure everyone's heard of the "Nyquist Theorem" - it basically says, that in digital audio, you have to record at double the sample rate, than the frequency you want to capture.
So, if you want to capture signals up to 22,000 Hz (human hearing goes to about 20 kHz, before that rolls off with age/damage to the ears), you record at 44.1 kHz.
If you need to record a dog whistle or the sound bats make, you can capture them at 96 kHz (or higher).
Edits at higher vs lower sample rates don't usually make a difference from my experience, as long as you cut at a zero crossing. If you zoom into a wave-form down to the sample level, you just see many "dots" (each sample) in a row that form your waveform. For 88.2 kHz, you just have twice as many dots/samples than for 44.1 kHz, and because they get played back so quickly at high sample rates, their frequency of playback is high enough to reproduce high frequencies in the recorded material... so, for 88.2 kHz, up to around 44,000 Hz.
If you also have speakers that can reproduce frequencies higher than 20 kHz (rare, but they exist), you can drive your dog (bat-)shit crazy
Of course humans can't hear frequencies this high at all... but the argument many made about higher frequencies was, that it just sounds "smoother" because the samples get played back in such fast succession and that aliasing is pushed out of the audible range. But as much as this may be true in theory, hardly anybody can reliably tell the difference, anyway.
But again... from an editing perspective, the sample rate doesn't really do much to keep audio clean, from my understanding. But it's also VERY possible, that I have a giant blind-spot when it comes to that (if so, sorry for blabbering on about this on here, anyway).
What a higher sample rate DOES help with, is latency. You can run lower latencies at 88.2/96 kHz than you can at 44.1/48 kHz. I think it has something to do with how quickly the buffer is read out, but I always get confused about that.
Anyway... I'll happily keep working at 44.1kHz/24-bit. If they'll ever make 32-bit converters, I'll upgrade and will record at 44.1/32-bit (and since the D8B can't deal with that, that's the time when it will have to go)... but I don't personally see a benefit in recording at higher sample rates.