Why continue into the beyond?

I have just returned from a very excellent 37th International Conference on High-Energy Physics. However, as splendid as the event itself was, it was in a sense bad news: No results which hint at anything substantial beyond the standard model, except for the usual suspect statistical fluctuations. This does not mean that there is nothing - we know there is more for many reasons. But in an increasingly frustrating sequence of years all our observational and experimental results keep pushing it beyond our reach. Even for me as a theorist there is just not enough substantial information to be able to do more than just vague speculation of what could be.

Nonetheless, I just wrote that I want to venture into this unknown beyond, and in force. Hence it is reasonable - in fact necessary - to pose the question: Why? If I do not know and have too little information, is there any chance to hit the right answer? The answer to this: Probably not. But...

Actually, there are two buts. One is simply curiosity. I am a theorist, and I can always pose the question how does something work, even without having a special application or situation in mind. Though this may just end up as nothing, it would not be the first time that the answer to a question has been discovered long before the question. In fact, the single most important building block of the standard-model, so-called Yang-Mills theory, has been discovered by theorists almost a decade before it was recognized to be the key to explain the experimental results.

But this is not the main reason for me to venture into this direction. The main reason has to do with the experience I made with Higgs physics - that despite appearance there is often a second layer to the theory. Such a second layer has in this case shifted the perception of how things we describe in theory correlate with the things we see in experiment. Since many proposed theories beyond the standard model, especially such as have caught my interest, are extensions of the Higgs of the standard model. It thus stands to reason that similar statements hold true in their cases. However, whether they hold true, and how they work cannot be fathomed without looking at theses theories. And that is what I want to do.

Why should one do this? Such subtle questions seem to be at first not really related to experiment. But understanding how a theory really works should also give us a better idea of what kind of observations such a theory can actually deliver. And now it becomes very interesting for an experiment. Since we do at the current time not know what to expect, we need to think about what we could expect. This is especially important as to look in every corner requires much more resources than available to us in the foreseeable future. Hence, any insights into what kind of experimental results a theory can yield is very important to select where to focus.

Of course, my research alone will not be sufficient to do this. Since it easily can be that I am looking at the 'wrong' theory, it would not be a good idea to put too much effort in it. But, when there are many theoreticians working on many theories, and many theories all say that it is a good idea to look into a particular direction: Then we have a guidance for where to look. Then there seems to be something special in this direction. And if not, then we have excluded a lot of theories in one go.

As one person in a discussion session (I could not figure out who precisely) has put it aptly at the conference: "The time of guaranteed discoveries is over.". This means that now that we have all pieces of the standard model, we cannot expect to find a new piece any time soon. All our indirect results even tell us that the next piece will be much harder to find. Hence, we are facing a situation as was last seen in physics in the second half of the 19th century and beginning 20th century: There are only some hints that something does not fit. And now we have to go looking, without knowing in advance how far we will have to walk. Or in which direction. This is probably more of an adventure than the last decades, where things where essentially happening on schedule. But is also requires more courage, since there will be much more dead ends (or worse) available.