Atomic force microscopy (AFM) is routinely used as a metrological tool among diverse scientific and engineering disciplines. A typical AFM, however, is intrinsically limited by low throughput and is inoperable under extreme conditions. Thus, this work attempts to provide an alternative with a conventional optical microscope (OM) by training a deep learning model to predict surface topography from surface OM images. The feasibility of our novel methodology is shown with germanium-on-nothing (GON) samples, which are self-assembled structures that undergo surface and sub-surface morphological transformations upon high-temperature annealing. Their transformed surface topographies are predicted based on OM-AFM correlation of 3 different surfaces, bearing an error of about 15% with 1.72× resolution upscale from OM to AFM. The OM-based approach brings about significant improvement in topography measurement throughput (equivalent to OM acquisition rate, up to 200 frames per second) and area (∼1 mm²). Furthermore, this method is operable even under extreme environments when an _in-situ_ measurement is impossible. Based on such competence, we also demonstrate the model’s simultaneous application in further specimen analysis, namely surface morphological classification and simulation of dynamic surfaces’ transformation.

Organic memristors are promising candidates for the flexible synaptic components of wearable intelligent systems. With heightened concerns for the environment, considerable effort has been made to develop organic transient memristors to realize eco-friendly flexible neural networks. However, in the transient neural networks, achieving flexible memristors with bio-realistic synaptic plasticity for energy efficient learning processes is still challenging. Here, we demonstrate a biodegradable and flexible polymer based memristor, suitable for the spike-dependent learning process. An electrochemical metallization phenomenon for the conductive nanofilament growth in a polymer medium of poly (vinyl alcohol) (PVA) is analyzed and a PVA based transient and flexible artificial synapse is developed. The developed device exhibits superior biodegradability and stable mechanical flexibility due to the high water solubility and excellent tensile strength of the PVA film, respectively. In addition, the developed flexible memristor is operated as a reliable synaptic device with optimized synaptic plasticity, which is ideal for artificial neural networks with the spike-dependent operations. The developed device is found to be effectively served as a reliable synaptic component with high energy efficiency in practical neural networks. This novel strategy for developing transient and flexible artificial synapses can be a fundamental platform for realizing eco-friendly wearable intelligent systems.Corresponding author(s) Email:   sinhlee@knu.ac.kr 

IntroductionReal world Time on Treatment (rwToT), also known as real world time to treatment discontinuation (rwTTD), is defined as the length of time observed in real world data (as distinct from controlled clinical trials) from initiation of a medication to discontinuation of that medication1,2. The ending of the treatment can be caused by adverse events, deaths, switches of treatment and loss of follow up. Because time to treatment discontinuation can be readily obtained from electronic medical records, this effectiveness endpoint is convenient to evaluate the efficacy of a drug that is already approved for public use3. It is often used as a surrogate effectiveness endpoint, showing high correlation to progression-free survival and moderate-to-high correlation to overall survival4,5. As rwTTD is an important metric for drug effectiveness, it is routinely reported during the post-clinical trial phase2,4,6–9.Calculation of rwTTD in patient population is often equivalent to constructing a (Kaplan-Meier) KM curve, with each point representing the proportion of patients that are still on treatment at a specific time point 1. Either the entire curve, or mean rwTTD, restricted mean10, or the time point at which a specific portion of the patients (e.g. , 50%) dropping treatment is of interest. Currently, there is no existing machine learning scheme established to predict such a curve, or the midpoint, as the vast majority of the machine learning models have been focused on predicting individuals’ behavior rather than population-level behavior. Such a machine learning scheme, if established, has many meaningful clinical applications. For instance, given observed clinical parameters and outcomes in clinical trials, how do we derive expected time-to-treatment in the real-world? Given the rwTTD for a drug on one patient population, how can we predict the rwTTD when applying this drug to another population (e.g. , for a different disease)?This study establishes a machine learning framework to infer population-wise rwTTD. We showed that population-wise curve prediction differs substantially from aggregating all individuals’ results. Our framework models the population-wise curve and is generic to diverse base-learners for predicting rwTTD. We demonstrated the effectiveness of this framework based on both simulated data and real world Electronic medical records (EMR) data for pembrolizumab-treated cancer populations7,11,12. The study opens a new direction of modeling population-level rwTTD, which has great values for directing post-clinical stage drug administrations. This machine learning scheme will also have meaningful implications to population-based predictions for other problems, as machine learning algorithms have so far been focused on predictions for individual samples.

Supplementary Materials: Figures S1, S2, S3, Equation S1

This Supporting Information includes: the detailed dimensions of the MRE valves, the dimensions of the four different MRE geometries investigated, and the detailed setup and parameters of the PID controllers used. Corresponding author Email:  sihan.wang@eng.ox.ac.uk 

Multiparametric magnetic resonance imaging (mpMRI) has emerged as a valuable diagnostic tool in prostate lesion assessment, for which many reports conclude the advantageous use of convolutional neural networks (CNNs) in prostate lesion detection and classification (PLDC). However, the network training inevitably involves prostate magnetic resonance (MR) images from multiple sites/cohorts. There always exists variation in scanning protocol among the cohorts, inducing significant changes in data distribution between source and target domains. This challenge has greatly limited clinical adoption on a large scale. Recent domain adaptation (DA) models can alleviate the inherent cross-site heterogeneity. Some could leverage cross-domain knowledge transfer using whole-slide images, without prior knowledge of lesion regions. In this paper, we propose a coarse mask-guided deep domain adaptation network (CMD²A-Net) in order to develop a fully automated framework for PLDC using multi-cohort images. No category or mask label is required from the target domain. A coarse segmentation module is trained to cover the possible lesion-related regions, so that attention maps can be generated to dedicate the local feature extraction of lesions within those regions. As a result, the features of both prostate lesion and region can be fused to align the robust features between the source and target domains. Experiments have been performed on 512 mpMRI sets from datasets of PROSTATEx (with 330 sets) and two cohorts, A (with 74 sets) and B (with 108 sets). Using the ensemble learning, our CMD²A-Net accomplishes an AUC of 0.921 in cohort A and 0.913 in cohort B, demonstrating its transferability from a large-scale public dataset PROSTATEx to our small-scale target domains. Our results and ablation study also support the CMD²A-Net’s effectiveness in lesion classification between benign or malignant, compared to the state-of-the-art models Corresponding author(s) Email:   kwokkw@hku.hk (K.W. Kwok), varv@hku.hk (V. Vardhanabhuti)

AbstractFabrication of structures in unstructured environments is a promising field to expand the application spaces of additive manufacturing (AM). One potential application is to add new components directly onto existing structures. In this paper, we developed a versatile, reconfigurable direct ink write (DIW) manufacturing method in tandem with a two-stage hybrid ink designed to fabricate high-strength, self-supporting parts in unconventional printing spaces, such as underneath a build surface or horizontally. Our two-stage hybrid DIW ink combines a photopolymer and a tough epoxy resin. The photopolymer can be cured rapidly to enable layer-by-layer printing complex structures. It also possesses adequate adhesion to allow the fabrication of large volume structures on a diversity of substrates including acrylic, wood, glass, aluminum, and concrete. The epoxy component can be cured after 72 hours in ambient conditions with further increased adhesion strengths. We demonstrated the capabilities of the reconfigurable DIW extrusion nozzle method to print complex structures in inverted and horizontal environments. Finally, via the addition of DIW-deposited conductive paths, we created a functional 3D printed structure capable of in-situ deformation monitoring. This work has the potential to be used for applications such as appending new parts to existing structures for increasing functionality, repair, and structure health monitoring.Corresponding author: H. Jerry Qi, qih@me.gatech.edu

The interest in soft pneumatic actuators has been growing rapidly in robotics, owing to the contact adaptability with the material softness. However, these actuators are mostly controlled by rigid electronic pneumatic valves, which can hardly be integrated into the robot itself, limiting its mobility and adaptability. Recent advances in soft or electronics-free valve designs provide the potential to achieve an integrated soft robotic system with reduced weight and rigidity. Nevertheless, the challenge in valve response remains open. To enable dynamic control of a soft pneumatic actuator, a fast-response proportional valve is needed. In this paper, we explored the potential of Ecoflex-based magnetorheological elastomer (MRE) membrane to create a proportional valve that can be used in the control of a soft robot made from the same silicone material. Experimental characterization shows that the proposed MRE valve (30 mm \(\times\) 30 mm \(\times\) 15 mm, 30 grams) can hold pressure up to 41.3 kPa and regulate the airflow in an analog manner. The valve is used to perform closed-loop Proportional-Integral-Differential (PID) control with 50 Hz on a soft pneumatic actuator and is able to control the pressure within the actuator chamber with a root-mean-square error of 0.05 kPa.   Corresponding author(s) Email:   sihan.wang@eng.ox.ac.uk  

AbstractMotile cilia move in an asymmetric pattern and implement a metachronal wave (MCW) to facilitate fluid movement in a viscous environment. Studies have been conducted to mimic MCW movement of motile cilia, but the fabrication process was too complicating or there were difficulties in accurately mimicking the shape of the cilia. To overcome these limitations, we introduce a self-assembly method to fabricate a reprogrammable magnetically actuated self-assembled (RMS) cilia array that can be reprogrammed by changing the magnetization direction through additional magnetization. Using the RMS cilia array, a unilateral cilia array (UCA) channel and a bilateral cilia array (BCA) channel were constructed, and the motion and fluid flow of the RMS cilia array were analyzed by applying different magnetic fields (strike magnetic field and rotating magnetic field). When a rotating magnetic field was applied to the UCA channel, a distinct MCW appeared. In the BCA channel test, fluid pumping was observed when a strike magnetic field, whereas fluid mixing was observed when a rotating magnetic field was applied. Based on these results, it is expected that the proposed RMS cilia array and magnetic field actuation method can be applied to lab-on-a-chip or microfluidic channels for fluid mixing and pumping.1. IntroductionCilia are hair-like, microtubule-based structures that have various distributions with a length of approximately 3–200 µm and an aspect ratio ranging from 10 to 100, depending on the location where they are found, and are divided into primary cilia and motile cilia.[1–5] Motile cilia can move objects or mix fluids by moving mucus or body fluids in the human body.[6,7] Among these motile cilia, the cilia that are found in the fallopian tube of the female reproductive system help the movement of the ovary, and the cilia existing in the lungs mix settled dust and bacteria through mucociliary clearance and move them out of the body.[8–10] The environment in which motile cilia move is normally filled with fluid with a low Reynolds number. In such an environment, the viscous force is generally more dominant than the inertial force, and has a significant influence on the fluid flow.[11–13] To be helpful in this environment, the cilium moves in an asymmetrical pattern comprising an effective stroke and a recovery stroke, creating a net fluid flow. In an effective stroke, the cilium moves in an arc that is fully stretched, while in the recovery stroke, the cilium returns to the starting point in a bent state as if swinging, which increases the moving area of the cilium.[14–16] In addition, when several cilia gather to form a cilia array, they move in a sequential pattern that forms a wave called the metachronal wave (MCW), which helps move the fluid faster and more efficiently because of their asymmetrical motion.[17–20]Several studies have reported mimicking the asymmetric motion of cilia and the MCW motion to efficiently pump or mix fluids in microfluidic devices with low Reynolds numbers.[21–23] To mimic cilia motion, many actuation methods have been used; actuation via a magnetic field is the most used method among them.[24–27] In addition, diverse manufacturing methods exist for magnetically actuated artificial cilia, and, the fabrication method using self-assembly has the advantages of simplicity and capability to mimic the appearance of natural cilia. However, it is difficult to program the magnetization direction, thus limiting the implementation of the MCW of the cilia.[28–31]Nevertheless, studies have been conducted to imitate the metachronal wave of cilia by fabricating artificial cilia using the molding method, which facilitates reprogramming.[32,33] Nelson formed a cilia array using a molding method and then reprogrammed the cilia array, which moved the cilia array to form an MCW.[34] Sitti fabricated micro-cilia using a mold, magnetized each cilium independently, and attached them to form a cilia array with the desired arrangement that implements the MCW.[35] However, these studies actuated the cilia array using only a rotating external magnetic field, and because the cilia array was fabricated using the molding method, several complex steps were required for making the cilia array.

This Supporting Information includes: _Supplementary text describing Preliminary Status Classifer, segmentation methods, model training and validation details; two supplementary tables, two supplementary figures and one supplementary video._ Corresponding author(s) Email:  _ hoychan@cityu.edu.hk; xinyueli@cityu.edu.hk; wenjli@cityu.edu.hk  _

More than 120 million mice and rats are used yearly for scientific purposes. While tracking their motion behaviors has been an essential issue for the past decade, present techniques, such as video-tracking and IMU-tracking have considerable problems, including requiring a complex setup or relatively large IMU modules that cause stress to the animals. Here, we introduce a wireless IoT motion sensor (i.e., weighing only 2 grams) that can be attached and carried by mice to collect motion data continuously for several days. We also introduce a combined segmentation method and an imbalanced learning process that are critical for enabling the recognition of common but random mouse behaviors (i.e., resting, walking, rearing, digging, eating, grooming, drinking water, and scratching) in cages with a macro-recall of 94.55%. Corresponding author(s) Email:  _ hoychan@cityu.edu.hk; xinyueli@cityu.edu.hk; wenjli@cityu.edu.hk  _

In existing surgery process, surgeons need to manually adjust the laparoscopes to provide a better field of view during operation, which may distract surgeons and slow down the surgery process. This paper presents a data-driven control method that uses a continuum laparoscope to adjust the field of view by tracking the surgical instruments. A Koopman-based system identification method is firstly applied to linearize the nonlinear system. Shifted Chebyshev polynomials are used to construct observation functions that transfer low-dimension observations to high-dimension ones. The Koopman operator is approximated using a finite-dimensional estimation method. An optimal controller is further developed according to the trained linear model. Furthermore, a learning-based pose estimation framework is designed to detect keypoints on surgical instruments and provide visual feedback for adjusting the laparoscope. Compared with other detection methods, the proposed scheme achieves a higher detection precision and provides more optional keypoints for tracking. Simulation and experiments validate the feasibility of the proposed control method. Experiment results show that the proposed method can automatically adjust the field of continuum laparoscope through tracking surgical instruments in a timely manner and the number of surgical tools is not limited.

While deep learning approaches are mostly developed for single images, in real world applications, images are often taken as a series to inform decision making. Due to hardware (memory) and software (algorithm) limitations, so far, no methods have been developed to integrate multiple images. In this study, we present an approach that seamlessly integrates deep learning and traditional machine learning, to study multiple images and score rheumatoid arthritis joint damage. This method allowed the quantification of joining space narrowing to approach the clinical upper limit. Beyond predictive performance, we integrated the multilevel interconnections across joints and damage types into the machine learning model and revealed the cross-regulation map of joint damages in rheumatoid arthritis. Corresponding author(s) Email:   hyangl@umich.edu or gyuanfan@umich.edu

How to prepare and submit your paper1) APPL Interactive Papers are prepared and submitted via Authorea. You can get started by using the submission template. (Click Use Template to get started). If you prefer, you can also start a document from scratch or upload an existing manuscript. If you want to do so, from your Authorea account, in the top right click the + button to see available options.2) We recommend to organize your article by including dedicated (sub)sections; please refer to the more general Applied Research guideline for authors for details on document structure. The Submission Template that we provide has the required document structure.3) Once you have prepared your document and are ready to submit it for consideration, submit it to the  APPL Interactive Papers Authorea Collection. (Click Submit and select your manuscript). You can also submit directly from the document by selecting "Submit for review" and finding the "APPL Interactive Papers" portal from the list.4) Make your document public, by publishing it as a preprint. Click Publish and follow the prompts. Choose a few suitable keywords; also select CC 4.0 is the copyright license.5) Submit a static version of your document through APPL's journal's editorial platform.  We recommend that you download/export the interactive article in Word, PDF, or LaTeX format and submit your manuscript files.

This Supporting Information includes information regarding the magnetic field of the actuator magnet, MR-LF-S (which has the same geometry as MR-LF and a soft compartment), and a table comparing MR-LF to other small-scale, flexible magnetic crawler robots.  Corresponding author email:   yong.kong@utah.edu

This supporting information includes supplemental figures, movies, additional results, and the key steps in the neural network training algorithm.   Corresponding author Email:   yyang60@jhu.edu (Y.Y.)     libome@tsinghua.edu (B.L.)

The cocktail party problem refers to a challenging process when the human sensory system tries to separate a specific voice from a loud mixture of background sound sources. The problem is much more demanding for machines and has become the holy grail in robotic hearing. Despite the many advances in noise suppression, the intrinsic information from the contaminated acoustic channel remains difficult to recover. Here we show a simple-yet-powerful laser-assisted audio system termed REAL (Robot Ear Accomplished by Laser) to probe the vibrations of sound-carrying surfaces (mask, throat and other nearby surfaces) in optical channel, which is intrinsically immune to acoustic background noises. Our results demonstrate that REAL can directly obtain the audio-frequency content from the laser without acoustic channel interference. The signals can be further transcribed into human-recognizable audio by exploiting the internal time and frequency correlations through memory-enabled neural networks. The REAL system would enable a new way in human-robot interaction. Xiaoping Hong  Email:    hongxp@sustech.edu.cn

Two-dimensional metal-organic frameworks (2D-MOFs) have been extensively studied as promising materials in the fields of eletrocatalysis, drug delivery, electronic devicese, etc. However, few studies have explored the application potential of 2D-MOFs in novel neuromorphic computing devices. In this work, we report an optoelectronic neuromorphic transistor based on a 2D-MOFs/polymer charge-trapping layer. We found that, the large specific surface area, stable crystal structure, and highly accessible active sites in 2D-MOFs make them excellent charge-trapping materials for our devices, which are beneficial for mimicking the memory and learning functions observed in the organism's nervous systems. Different types of synaptic behaviors have been realized in our 2D-MOFs-based neuromorphic devices under stimuli signal, e.g., paired-pulse facilitation, excitatory post-synaptic current, short-term memory, and long-term memory. More interestingly, emotion-adjustable learning behavior was realized by changing the value of the source-drain voltage. This work can shed light on the application of 2D-MOFs in neuromorphic computing and will contribute to the further development of neuromorphic computing devices. Corresponding authors Email:  huangjia@tongji.edu.cn (Jia Huang)                                                                1610419@tongji.edu.cn (Shilei Dai)