publications

Probabilistic world models increase data efficiency of model-based reinforcement learning (MBRL) by guiding the policy with their epistemic uncertainty to improve exploration and acquire new samples. Moreover, the uncertainty-aware learning procedures in probabilistic approaches lead to robust policies that are less sensitive to noisy observations compared to uncertainty-unaware solutions. We propose to combine trajectory sampling and deep Gaussian covariance network (DGCN) for a data-efficient solution to MBRL problems in an optimal control setting. We compare trajectory sampling with density-based approximation for uncertainty propagation using three different probabilistic world models; Gaussian processes (GPs), Bayesian neural networks (BNNs), and DGCNs. We provide empirical evidence using four different well-known test environments, that our method improves the sample-efficiency over other combinations of uncertainty propagation methods and probabilistic models. During our tests, we place particular emphasis on the robustness of the learned policies with respect to noisy initial states.

In materials science, traditional techniques for analyzing layered structures are essential for obtaining information about local structure, electronic properties and chemical states. While valuable, these methods often require high vacuum environments and have limited depth profiling capabilities. The grazing exit x-ray absorption near-edge structure (GE-XANES) technique addresses these limitations by providing depth-resolved insight at ambient conditions, facilitating in situ material analysis without special sample preparation. However, GE-XANES is limited by long data acquisition times, which hinders its practicality for various applications. To overcome this, we have incorporated Bayesian optimization (BO) into the GE-XANES data acquisition process. This innovative approach potentially reduces measurement time by a factor of 50. We have used a standard GE-XANES experiment, which serve as reference, to validate the effectiveness and accuracy of the BO-informed experimental setup. Our results show that this optimized approach maintains data quality while significantly improving efficiency, making GE-XANES more accessible to a wider range of materials science applications.

Real-world applications of machine learning models are often subject to legal or policy-based regulations. Some of these regulations require ensuring the validity of the model, i.e., the approximation error being smaller than a threshold. A global metric is generally too insensitive to determine the validity of a specific prediction, whereas evaluating local validity is costly since it requires gathering additional data. We propose learning the model error to acquire a local validity estimate while reducing the amount of required data through active learning. Using model validation benchmarks, we provide empirical evidence that the proposed method can lead to an error model with sufficient discriminative properties using a relatively small amount of data. Furthermore, an increased sensitivity to local changes of the validity bounds compared to alternative approaches is demonstrated.

Surrogate models based on machine learning methods have become an important part of modern engineering to replace costly computer simulations. The data used for creating a surrogate model are essential for the model accuracy and often restricted due to cost and time constraints. Adaptive sampling strategies have been shown to reduce the number of samples needed to create an accurate model. This paper proposes a new sampling strategy for global fit called Gradient and Uncertainty Enhanced Sequential Sampling (GUESS). The acquisition function uses two terms: the predictive posterior uncertainty of the surrogate model for exploration of unseen regions and a weighted approximation of the second and higher-order Taylor expansion values for exploitation. Although various sampling strategies have been proposed so far, the selection of a suitable method is not trivial. Therefore, we compared our proposed strategy to 9 adaptive sampling strategies for global surrogate modeling, based on 26 different 1 to 8-dimensional deterministic benchmarks functions. Results show that GUESS achieved on average the highest sample efficiency compared to other surrogate-based strategies on the tested examples. An ablation study considering the behavior of GUESS in higher dimensions and the importance of surrogate choice is also presented.

Background: Offloading is the cornerstone of treatment of plantar diabetic foot ulcers. It limits mobility with consequent psychological and cardiovascular side effects, and if devices are removed, healing is delayed.Methods:We developed three non-removable techniques with increasing offloading potential (multilayer felt sole, felt-fiberglass sole, or total contact casts with ventral windows) and sensors built within. Smartwatch and web apps displayed pressure, temperature, humidity, and steps. They alerted patients, staff, and a telemedicine center when pressure limits (125 kPa) were exceeded. Patients were advised to walk as much as they had done before the ulcer episode. To evaluate the potential of this intervention, we enrolled 20 ambulatory patients in a randomized clinical trial. The control group used the same offloading and monitoring system, but neither patients nor therapists received any information or warnings.Results:Three patients withdrew consent. The median time to healing of ulcers was significantly shorter in the intervention group compared with controls, 40.5 (95% confidence interval [CI] = 28-not applicable [NA]) versus 266.0 (95% CI = 179-NA) days (P = .037), and increasing ulcer area was observed less frequently during study visits (7.9% vs 29.7%, P = .033). A reduction of wound area by 50% was reached at a median of 10.2 (95% CI = 7.25-NA) versus 19.1 (95% CI = 13.36-NA) days (P = .2). Participants walked an average of 1875 (SD = 1590) steps per day in intervention group and 1806 (SD = 1391) in the control group.Conclusions:Sensor-assisted wound therapy may allow rapid closure of plantar foot ulcers while maintaining patient’s mobility during ulcer therapy.

The design uncertainty is often ignored, although there are various random factors in any system such as production tolerances, operational conditions and so on. The number of experi- ments required for the uncertainty quantification and optimization of a design is often so high that they become inapplicable to complicated systems. Safety factors are used instead for a fast but implicit assessment of design reliability. Not only are these unintuitive measures, but they are also uninformative regarding the design robustness. Moreover, safety factors need to be cal- ibrated for each new design, which requires one or more reliability assessments, thus relocating the cost of accounting for uncertainty instead of reducing it. Surrogate model-based solutions are often used to accelerate the design process by reducing the number of required experiments. Adaptive experiment schemes are proposed to further reduce the required amount of data by concentrating on the important domains of the approximated responses. However, most of these methods only consider a specific combination of methods regarding surrogate modelling and uncertainty quantification. This work presents an adaptive model refinement framework for the surrogate-based solution of reliability assessment and multi-objective reliability-based robust design optimization problems, which aims to maintain the freedom of surrogate choice since there is no single machine learning algorithm, which outperforms others in all problems

Optimizing the reliability and the robustness of a design is important but often unaffordable due to high sample requirements. Surrogate models based on statistical and machine learning methods are used to increase the sample efficiency. However, for higher dimensional or multi-modal systems, surrogate models may also require a large amount of samples to achieve good results. We propose a sequential sampling strategy for the surrogate based solution of multi-objective reliability based robust design optimization problems. Proposed local Latin hypercube refinement (LoLHR) strategy is model-agnostic and can be combined with any surrogate model because there is no free lunch but possibly a budget one. The proposed method is compared to stationary sampling as well as other proposed strategies from the literature. Gaussian process and support vector regression are both used as surrogate models. Empirical evidence is presented, showing that LoLHR achieves on average better results compared to other surrogate based strategies on the tested examples.

The lack of anomaly detection methods during mechanized tunnelling can cause financial loss and deficits in drilling time. On-site excavation requires hard obstacles to be recognized prior to drilling in order to avoid damaging the tunnel boring machine and to adjust the propagation velocity. The efficiency of the structural anomaly detection can be increased with intelligent optimization techniques and machine learning. In this research, the anomaly in a simple structure is detected by comparing the experimental measurements of the structural vibrations with numerical simulations using parameter estimation methods.

Reliability analysis of non-linear and multimodal limit state functions is a computationally burdensome task, especially if the sampling based strategies are followed. Response surface methods with various adaptive sampling schemes are previously developed to greatly reduce the required number of sample points] but these methods either do not account for the multimodal limit state function or use linear regression based designs of experiments (i.e. D-Optimal) which do not harmonize as well with other response surface methods. We propose combining an unsupervised learning algorithm (DBSCAN) for handling the multimodality with a Latin hypercube design based approach and adapting the process to reliability analysis efficiently. Anisotropic universal Kriging is used for surrogate modeling in combination with directional sampling for limit state functions with a low number of parameters n ≤ 12 and Monte-Carlo method for n > 12.

Growing popularity of probabilistic and stochastic optimization methods in engineering applications has vastly increased the number of required sampling points to obtain a solution. Depending on the complexity of the underlying physical model, this often proves to be a computationally burdensome challenge. In order to overcome this challenge, one possible approach is to use surrogate models, which approximate the responses of the physical model in a given variable subspace. In the past years, many different metamodeling algorithms like Gaussian process (Kriging), moving least squares, radial basis functions, regression neural networks and support vector regression have been proposed as an alternative to ordinary least squares (polynomial regression). The choice of the best metamodeling algorithm for any application is not a trivial task. Although there have been previous attempts to compare at least some of these methods to some extent, there is only a small number of publications comparing all of these algorithms over a large number of multidimensional functions with varying characteristics. In this paper, a comparison of the aforementioned methods is carried out. Using well-known analytical test functions for optimization, this paper aims to shed some light on the question, which algorithm performs best under which conditions. Apart from the structure of analytical test functions, the influence of the number of sampling points and amount of noise in the observations on the performance of metamodeling algorithms is investigated, since these are often two very important constraints regarding industrial applications.