Twitter has attracted several users to share and propagate newest data, leading to giant volumes of knowledge made on a daily basis. However, several applications in data Retrieval (IR) and tongue process (NLP) suffer severely from the clanging and short nature of tweets. During this paper, we have a tendency to propose a unique framework for tweet segmentation in a very batch mode, known as HybridSeg. By rending tweets into meaning segments, the linguistics or context data is well preserved

more »

... simply extracted by the downstream applications. HybridSeg finds the optimum segmentation of a tweet by increasing the ad of the viscosity immeasurable its candidate segments. The viscosity score considers the likelihood of a section being a phrase in English (i.e., international context) and also the likelihood of a section being a phrase inside the batch of tweets (i.e., native context). For the latter, we have a tendency to propose and judge 2 models to derive native context by considering the linguistic options and term-dependency in a very batch of tweets, severally. HybridSeg is additionally designed to iteratively learn from assured segments as pseudo feedback. Experiments on 2 tweet information sets show that tweet segmentation quality is considerably improved by learning each international and native contexts compared with victimization international context alone. Through analysis and comparison, we have a tendency to show that native linguistic options area unit a lot of reliable for learning native context compared with term-dependency. As associate degree application, we have a tendency to show that top accuracy is achieved in named entity recognition by applying segment-based part-of-speech (POS) tagging.