Is there an easier way to select a single line?

I am using the new layout, which consolidated breakpoints and line numbers in the same column. I feel this was a good change, as the breakpoint gutter, and line number gutter, were small while next to each other. But I often use the line numbers as a way to select one or more lines. This new change makes it difficult to select a single line, as one has to click and drag within the scope of a single line. Or drag across multiple lines, and drag back down to a single line.

Is there a way to only toggle a breakpoint when a modifier is pressed? Or vice versa, only select line(s) while a modifier key is pressed? A behavioral example being, when clicking a line number, the respective line is selected, while if ALT is held while clicking a line number, a breakpoint is toggled.