Subjects: Computer Science >> Natural Language Understanding and Machine Translation submitted time 2023-07-12
Abstract： The Rotary Position Embeddings(RoPE) is widely used in open-source large language models such as LLAMA. In the original paper, the formula derivation uses complex functions. In this Paper, I derive PoPE’s formulas again with linear algebra, hoping to better understand this method.
Peer Review Status:
Awaiting Review
Subjects: Mathematics >> Mathematics （General） submitted time 2022-10-10
Abstract： In weibo APP, there are many almost same images whose only difference are watermark and resolution. In order to find out the most similar image efficiently, this paper proposes a algorithm named multi-level fingerprint, which contains 5 character strings and 3 vectors. On a dataset of 1 million images from WEIBO APP, multi-level fingerprint achieves a precision 97.69% and QPS 345.
Peer Review Status:Awaiting Review
Subjects: Computer Science >> Natural Language Understanding and Machine Translation submitted time 2021-10-20
Abstract： BERT is the most popular natural language processing(NLP) model In the recent 3 years. This paper presents its mathematic formulas in detail.
Peer Review Status:Awaiting Review
Subjects: Mathematics >> Computational Mathematics. submitted time 2020-10-19
Abstract： Before entering into a recommender system, an entity name must be embedded into a vector. Some popular models, such as word2vec, are based on the principle “words which are in the same syntactic position should embedded into similar vectors”. However, sequence of entity names has no syntactic structure, which led to the low quality of name vectors. Based on the principle “neighbouring names should embedded into similar vectors”, this paper proposes a novel algorithm named name2vec. Name2vec has new features: vector length equals 1, relative weight which has solved the low frequency problem, optimization objective function is mean square error rather than cross entropy. The quality of embedding is measured by the similarity of entity names. On there datasets from WEIBO.COM, name2vec has a better performance than word2veec.
Peer Review Status:Awaiting Review
Hits
Hits
Downloads
Comment