Neural Sequence-to-Sequence Model for Explaining Non-Standard English Expressions

Table of Contents

1. Introduction

Natural language processing has traditionally focused on Standard English in formal contexts, leaving non-standard expressions largely unaddressed. This research tackles the critical challenge of automatically explaining emerging non-standard English words and phrases found in social media and informal communication.

The rapid evolution of language in digital spaces creates a significant gap in NLP capabilities. While traditional dictionary-based approaches struggle with coverage issues, our neural sequence-to-sequence model provides a dynamic solution for understanding contextual meaning of slang and informal expressions.

2. Related Work

Previous approaches to non-standard language processing primarily relied on dictionary lookups and static resources. Burfoot and Baldwin (2009) used Wiktionary for satire detection, while Wang and McKeown (2010) employed a 5K-term slang dictionary for Wikipedia vandalism detection. These methods face fundamental limitations in handling the rapid evolution of language in social media environments.

Recent advances in word embeddings by Noraset (2016) showed promise but lacked contextual sensitivity. Our approach builds upon sequence-to-sequence architectures pioneered by Sutskever et al. (2014), adapting them specifically for the challenges of non-standard language explanation.

3. Methodology

3.1 Dual Encoder Architecture

The core innovation of our approach is a dual encoder system that processes both context and target expressions separately. The architecture consists of:

• Word-level encoder for contextual understanding
• Character-level encoder for target expression analysis
• Attention mechanism for focused explanation generation

3.2 Character-Level Encoding

Character-level processing enables handling of out-of-vocabulary words and morphological variations common in non-standard English. The character encoder uses LSTM units to process input sequences character by character:

$h_t = \text{LSTM}(x_t, h_{t-1})$

where $x_t$ represents the character at position $t$, and $h_t$ is the hidden state.

3.3 Attention Mechanism

The attention mechanism allows the model to focus on relevant parts of the input sequence when generating explanations. The attention weights are computed as:

$\alpha_{ti} = \frac{\exp(\text{score}(h_t, \bar{h}_i))}{\sum_{j=1}^{T_x} \exp(\text{score}(h_t, \bar{h}_j))}$

where $h_t$ is the decoder hidden state and $\bar{h}_i$ are encoder hidden states.

4. Experimental Results

4.1 Dataset and Evaluation

We collected 15 years of crowdsourced data from UrbanDictionary.com, comprising millions of non-standard English definitions and usage examples. The dataset was split into training (80%), validation (10%), and test (10%) sets.

Evaluation metrics included BLEU scores for definition quality and human evaluation for plausibility assessment. The model was tested on both seen and unseen non-standard expressions to measure generalization capability.

4.2 Performance Comparison

Our dual encoder model significantly outperformed baseline approaches including standard attentive LSTMs and dictionary lookup methods. Key results include:

• 35% improvement in BLEU scores over baseline LSTM
• 72% accuracy in human evaluation for plausibility
• Successful explanation generation for 68% of unseen expressions

5. Conclusion and Future Work

Our research demonstrates that neural sequence-to-sequence models can effectively generate explanations for non-standard English expressions. The dual encoder architecture provides a robust framework for handling the contextual nature of slang and informal language.

Future directions include expanding to multilingual non-standard expressions, incorporating temporal dynamics of language evolution, and developing real-time explanation systems for social media platforms.

6. Technical Analysis

Core Insight

This research fundamentally challenges the dictionary-based paradigm that has dominated non-standard language processing. The authors recognize that slang isn't just vocabulary—it's contextual performance. Their dual-encoder approach treats explanation as translation between linguistic registers, a perspective that aligns with sociolinguistic theories of code-switching and register variation.

Logical Flow

The argument progresses from identifying the coverage limitations of static dictionaries to proposing a generative solution. The logical chain is compelling: if slang evolves too rapidly for manual curation, and if meaning is context-dependent, then the solution must be both generative and context-aware. The dual encoder architecture elegantly addresses both requirements.

Strengths & Flaws

Strengths: The scale of Urban Dictionary data provides unprecedented training coverage. The character-level encoder cleverly handles morphological creativity in slang formation. The attention mechanism provides interpretability—we can see which context words influence explanations.

Flaws: The model likely struggles with highly contextual or ironic usage where surface-level patterns mislead. Like many neural approaches, it may inherit biases from training data—Urban Dictionary entries vary widely in quality and may contain offensive content. The evaluation focuses on technical metrics rather than real-world utility.

Actionable Insights

For practitioners: This technology could revolutionize content moderation, making platforms more responsive to evolving harmful speech patterns. For educators: Imagine tools that help students decode internet slang while maintaining academic writing standards. The architecture itself is transferable—similar approaches could explain technical jargon or regional dialects.

The research echoes architectural patterns seen in successful multimodal systems like CLIP (Radford et al., 2021), where separate encoders for different modalities create richer representations. However, the application to register translation rather than cross-modal understanding is novel and promising.

Analysis Framework Example

Case Study: Explaining "sus" in Context

Input: "That explanation seems pretty sus to me."
Model Processing:
- Word encoder analyzes full sentence context
- Character encoder processes "sus"
- Attention identifies "explanation" and "seems" as key context
Output: "suspicious or untrustworthy"

This demonstrates how the model leverages both the target expression's form and its syntactic/semantic context to generate appropriate explanations.

Future Applications

Beyond the immediate application of slang explanation, this technology could enable:

• Real-time translation between formal and informal registers
• Adaptive educational tools for language learners
• Enhanced content moderation systems that understand evolving harmful speech patterns
• Cross-cultural communication aids for global digital spaces

7. References

1. Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27.
2. Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., ... & Sutskever, I. (2021). Learning transferable visual models from natural language supervision. International Conference on Machine Learning.
3. Burfoot, C., & Baldwin, T. (2009). Automatic satire detection: Are you having a laugh?. Proceedings of the ACL-IJCNLP 2009 conference short papers.
4. Wang, W. Y., & McKeown, K. (2010). Got you!: automatic vandalism detection in wikipedia with web-based shallow syntactic-semantic modeling. Proceedings of the 23rd International Conference on Computational Linguistics.
5. Noraset, T., Liang, C., Birnbaum, L., & Downey, D. (2017). Definition modeling: Learning to define word embeddings in natural language. Thirty-First AAAI Conference on Artificial Intelligence.