Wei Wang

Despite the demonstrated empirical efficacy of prompt tuning to adapt a pretrained language model for a new task, the theoretical underpinnings of the difference between" tuning parameters before the input" against" the tuning of model weights" are limited. We thus take one of the first steps to understand the role of soft-prompt tuning for transformer-based architectures. By considering a general purpose architecture, we analyze prompt tuning from the lens of both: universal approximation and limitations with finite-depth fixed-weight pretrained transformers for continuous-valued functions. Our universality result guarantees the existence of a strong transformer with a prompt to approximate any sequence-to-sequence function in the set of Lipschitz functions. The limitations of prompt tuning for limited-depth transformers are first proved by constructing a set of datasets, that cannot be memorized by a prompt of any length for a given single encoder layer. We also provide a lower bound on the required number of tunable prompt parameters and compare the result with the number of parameters required for a low-rank update (based on LoRA) for a single-layer setting. We finally extend our analysis to multi-layer settings by providing sufficient conditions under which the transformer can at best learn datasets from invertible functions only. Our theoretical claims are also corroborated by empirical results.

BACKGROUNDMany studies have explored the relationship between depression and metabolic syndrome (MetS), especially in older people. China has entered an aging society. However, there are still few studies on the elderly in Chinese communities.AIMTo investigate the incidence and risk factors of depression in MetS patients in mainland China and to construct a predictive model.METHODSData from four waves of the China Health and Retirement Longitudinal Study were selected, and middle-aged and elderly patients with MetS (n= 2533) were included based on the first wave. According to the center for epidemiological survey-depression scale (CESD), participants with MetS were divided into depression (n= 938) and non-depression groups (n= 1595), and factors related to depression were screened out. Subsequently, the 2-, 4-, and 7-year follow-up data were analyzed, and a prediction model for …

The defense of Use-After-Free (UAF) exploits generally could be guaranteed via static or dynamic analysis, however, both of which are restricted to intrinsic deficiency. The static analysis has limitations in loop handling, optimization of memory representation and constructing a satisfactory test input to cover all execution paths. While the lack of maintenance of pointer information in dynamic analysis may lead to defects that cannot accurately identify the relationship between pointers and memory. In order to successfully exploit a UAF vulnerability, attackers need to reference freed memory. However, main existing schemes barely defend all types of UAF exploits because of the incomplete check of pointers. To solve this problem, we propose UAF-GUARD to defend against the UAF exploits via fine-grained memory permission management. Specially, we design two key data structures to enable the fine-grained …

ChatGPT, an artificial intelligence (AI)-powered chatbot developed by OpenAI, is creating a buzz across all occupational sectors. Its name comes from its basis in the Generative Pretrained Transformer (GPT) language model. ChatGPT’s most promising feature is its ability to offer human-like responses to text input using deep learning techniques at a level far superior to any other AI model. Its rapid integration in various industries signals the public’s burgeoning reliance on AI technology. Thus, it is essential to critically evaluate ChatGPT’s potential impacts on academic clinical and translational medicine research.

Anchor link prediction exacerbates the risk of privacy leakage via the de-anonymization of social network data. Embedding-based methods for anchor link prediction are limited by the excessive similarity of the associated nodes in a latent feature space and the variation between latent feature spaces caused by the semantics of different networks. In this article, we propose a novel method which reduces the impact of semantic discrepancies between different networks in the latent feature space. The proposed method consists of two phases. First, graph embedding focuses on the network structural roles of nodes and increases the distinction between the associated nodes in the embedding space. Second, a federated adversarial learning framework which performs graph embedding on each social network and an adversarial learning model on the server according to the observable anchor links is used to associate …

Recently, an exciting experimental conclusion in Li et al. (Knowl Inf Syst 62(2):611–637, ) about measures of uncertainty for knowledge bases has attracted great research interest for many scholars. However, these efforts lack solid theoretical interpretations for the experimental conclusion. The main limitation of their research is that the final experimental conclusions are only derived from experiments on three datasets, which makes it still unknown whether the conclusion is universal. In our work, we first review the mathematical theories, definitions, and tools for measuring the uncertainty of knowledge bases. Then, we provide a series of rigorous theoretical proofs to reveal the reasons for the superiority of using the knowledge amount of knowledge structure to measure the uncertainty of the knowledge bases. Combining with experiment results, we verify that knowledge amount has much better performance for …

This paper investigates the attitude tracking control problem of rigid spacecraft with inertia parameter identification. Based on the relative attitude and angular velocity error dynamics, a basic adaptive backstepping based attitude tracking control scheme is firstly designed such that asymptotic attitude tracking can be achieved. However, the parameter identification error cannot decay to zero if the persistent excitation (PE) condition is not satisfied. To solve this issue, a filtering‐based concurrent learning adaptive backstepping control scheme is then proposed, by incorporating torque filtering technique with concurrent learning technique. A more mild rank condition, which consists of some collectable historical data, is provided to guarantee the convergence of parameter identification error. In addition, a valid data collection algorithm is given. It should be mentioned that a distinctive feature of the proposed filtering …

We present a deep-learning-based platform, MIND-S, for protein post-translational modification (PTM) predictions. MIND-S employs a multi-head attention and graph neural network and assembles a 15-fold ensemble model in a multi-label strategy to enable simultaneous prediction of multiple PTMs with high performance and computation efficiency. MIND-S also features an interpretation module, which provides the relevance of each amino acid for making the predictions and is validated with known motifs. The interpretation module also captures PTM patterns without any supervision. Furthermore, MIND-S enables examination of mutation effects on PTMs. We document a workflow, its applications to 26 types of PTMs of two datasets consisting of ∼50,000 proteins, and an example of MIND-S identifying a PTM-interrupting SNP with validation from biological data. We also include use case analyses of targeted …

As influencers play considerable roles in social media marketing, companies increase the budget for influencer marketing. Hiring effective influencers is crucial in social influencer marketing, but it is challenging to find the right influencers among hundreds of millions of social media users. In this paper, we propose InfluencerRank that ranks influencers by their effectiveness based on their posting behaviors and social relations over time. To represent the posting behaviors and social relations, the graph convolutional neural networks are applied to model influencers with heterogeneous networks during different historical periods. By learning the network structure with the embedded node features, InfluencerRank can derive informative representations for influencers at each period. An attentive recurrent neural network finally distinguishes highly effective influencers from other influencers by capturing the knowledge of the dynamics of influencer representations over time. Extensive experiments have been conducted on an Instagram dataset that consists of 18,397 influencers with their 2,952,075 posts published within 12 months. The experimental results demonstrate that InfluencerRank outperforms existing baseline methods. An in-depth analysis further reveals that all of our proposed features and model components are beneficial to discover effective influencers.

Manufacturing data analytics tasks are traditionally undertaken with Mechanism Models (MMs), which are domain-specific mathematical equations modeling the underlying physical or chemical processes of the tasks. Recently, Deep Learning (DL) has been increasingly applied to manufacturing. MMs and DL have their individual pros and cons, motivating the development of Mechanism-guided Deep Learning Models (MDLMs) that combine the two. Existing MDLMs are often tailored to specific tasks or types of MMs, and can fail to effectively 1) utilize interconnections of multiple input examples, 2) adaptively self-correct prediction errors with error bounding, and 3) ensemble multiple MMs. In this work, we propose a generic, task-agnostic MDLM framework that can embed one or more MMs in deep networks, and address the 3 aforementioned issues. We present 2 diverse use cases where we experimentally …

An adjustable bed includes a bed frame supporting a plurality of platforms having at least a head platform and a back platform, a back lifting assembly, a foot lifting assembly, a base frame pivotally and detachably connected to the bed frame, a bed frame tilting actuator pivotally connected to the bed frame and the base frame for operably adjusting the bed frame from the horizontal position to the sloping position relative to the base frame, or vice versa. The adjustable bed also includes a head platform tilting actuator pivotally connected to the head platform and the back platform for operably adjusting the head platform in a tilting position or a flat position relative to the back platform.

There are a large number of solutions for big data processing in the Internet of Things (IoT) environments, among which the IoT cloud infrastructure is one of the most mature solutions. Typically, modern IoT cloud infrastructures have different kinds of configuration options. The diversity of configurations leads to frequent software configuration errors. Generally, troubleshooting configuration errors relies on finding the mapping relationship between configuration options in the documents (e.g., official manuals) and their read sites in the source code. Most current works still manually extract configuration read sites. Automated methods are not always interchangeable and they incur considerable time overheads and low extraction rates.In this paper, we propose CRSExtractor, an automatic technique for extracting configuration read sites based on intra-procedural analysis. Using our technique, configuration option read …

Automated driving systems (ADSs) conceive an efficient and safe way of driving. The safety of ADSs depends on a precise object detector that needs to be upgraded continuously facing various environments. Massive annotations are required to utilize collected images of surroundings through vehicles and accommodate new environments. Auto labelling is one approach to alleviate such dilemma. To this end, we propose a novel Weakly Supervised Object Localization (WSOL) method which can localize objects precisely without detection annotations. This paper proposed Soft Guidance Module (SGM), Channel Erasing Module (CEM) and incorporate them into a multi-flow framework allowing the two mutually beneficial. Finally, experiments and visualizations are performed to evaluate our method on Stanford Cars, ILSVRC 2016 and CUB-200-2011 datasets.

Diabetic retinopathy (DR) is a complication of diabetic patients and a significant cause of blindness globally among the working population (Antonetti et al., 2021). There are 451 million suffering from DR in the world, and this is projected to increase to 639 million in 2045 (Cho et al., 2018). In diabetics, blood is provided to all retina layers through micro blood vessels that are sensitive to unrestricted blood sugar levels. DR may cause no symptoms or only mild vision problems at first, but it can cause blindness eventually. When substantial glucose or fructose is collected in the blood, blood vessels begin to collapse due to insufficient oxygen supply to the cells. Occlusion in these blood vessels can cause serious eye damage. As a result, metabolic rate decreases, and abnormal blood vessels accumulate in DR (Dai et al., 2021). Microaneurysms (MAs) are the early signs of DR, which

PurposeTo describe an artificial intelligence platform that detects thyroid eye disease (TED).DesignDevelopment of a deep learning model.Methods1944 photographs from a clinical database were used to train a deep learning model. 344 additional images (‘test set’) were used to calculate performance metrics. Receiver operating characteristic, precision–recall curves and heatmaps were generated. From the test set, 50 images were randomly selected (‘survey set’) and used to compare model performance with ophthalmologist performance. 222 images obtained from a separate clinical database were used to assess model recall and to quantitate model performance with respect to disease stage and grade.ResultsThe model achieved test set accuracy of 89.2%, specificity 86.9%, recall 93.4%, precision 79.7% and an F1 score of 86.0%. Heatmaps demonstrated that the model identified pixels corresponding to …

Structure prediction tasks such as event extraction require an in-depth understanding of the output structure and sub-task dependencies, thus they still heavily rely on task-specific training data to obtain reasonable performance. Due to the high cost of human annotation, low-resource event extraction, which requires minimal human cost, is urgently needed in real-world information extraction applications. We propose to synthesize data instances given limited seed demonstrations to boost low-resource event extraction performance. We propose STAR, a structure-to-text data generation method that first generates complicated event structures (Y) and then generates input passages (X), all with Large Language Models. We design fine-grained step-by-step instructions and the error cases and quality issues identified through self-reflection can be self-refined. Our experiments indicate that data generated by STAR can significantly improve the low-resource event extraction performance and they are even more effective than human-curated data points in some cases.

There are four equally important major building blocks of life: nucleic acids (DNA and RNA), proteins, carbohydrates (glycans), and lipids. The first two are also known as the first and second alphabets of biology, following the principle of the ‘‘central dogma” of transcription (DNA to RNA) and translation (RNA to protein). However, the latter two crucial components, glycans and lipids, are missing from biology’s central dogma. Regarding the communication between glycans and lipids, there may be a yet-to-bediscovered law: Does a paracentral dogma exist? This commentary focuses on glycans, the third alphabet of life, and their role in the sociomateriality of the cell, which provides a novel dimension of medical science—glycomedicine. This is an allied new discipline that employs glycomics approaches with the aim of better targeting disease diagnostics, as well as drug discovery, prescription choice, and dosing …

As a primary goal of predictive analytics, uplift modeling is used to estimate what impact a specific action or treatment will have on an outcome. In convention, the treatment is evaluated as a success once the buyer has purchased following the treatment, regardless of the kinds of treatments and the corresponding cost. Obviously, it cannot be classified as a binary classification problem. Therefore, we extend the ordinary uplift model to support multi-treatments tasks. In order to reconcile this aspect of interpretability with tree-based models, we use random forest (RF) as our base model. We present Gross Merchandise Value (GMV)-based RF for uplift modeling (GRFlift): an uplift model, where typical commercial evaluation GMV is designed as novel tree splitting criteria to directly quantify the uplift achievement. A targeted regularization term is also designed to adjust the splitting distribution differences. The splitting …

This paper considers the one-way trading problem with fixed costs where the trader can only trade in one direction throughout, either sell or buy, and he only knows limited information on price fluctuations beforehand. We construct a robust optimization model based on Savage's regret criterion, in order to find the online trading policy that minimizes the worst-case regret. However, it is very difficult to obtain analytical results if the trading horizon is relatively long, due to the discontinuity in the trader's objective function caused by the fixed cost. Thus we propose to solve the alternative problem with prepaid trading opportunities, which is not only a satisfactory approximation of the original one, but also a realistic problem with many practical applications, such as in the stock or future market. The optimal online trading policy of the new problem can be easily found based on the existing results of the one-way trading …

A47C31/00—Details or accessories for chairs, beds, or the like, not provided for in other groups of this subclass, eg upholstery fasteners, mattress protectors, stretching devices for mattress nets