https://www.depends-on-the-definition.com/Recent content on Depends on the definitionHugo -- gohugo.ioen© depends-on-the-definition 2017-2022Sun, 19 Feb 2023 00:00:00 +0000https://www.depends-on-the-definition.com/causal-graphs-and-deconfounding/Sun, 19 Feb 2023 00:00:00 +0000https://www.depends-on-the-definition.com/causal-graphs-and-deconfounding/I read into causal inference recently and since I didn’t really have a use-case for it right now, I played around with some data and some causal graphs. In this article, I looked at some causal graphs from the “Book of Why” Chapter 4 by Judea Pearl and Dana Mackenzie and created simulated data based of them.https://www.depends-on-the-definition.com/shapley-values-from-scratch/Tue, 19 Jul 2022 00:00:00 +0000https://www.depends-on-the-definition.com/shapley-values-from-scratch/The shapley value is a popular and useful method to explain machine learning models. The shapley value of a feature is the average contribution of a feature value to the prediction. In this article I’ll show you how to compute shapley values from scratch.https://www.depends-on-the-definition.com/how-to-add-new-tokens-to-huggingface-transformers/Thu, 12 May 2022 00:00:00 +0000https://www.depends-on-the-definition.com/how-to-add-new-tokens-to-huggingface-transformers/In this short article, you’ll learn how to add new tokens to the vocabulary of a huggingface transformer model.https://www.depends-on-the-definition.com/test-error-messages-with-pytest/Wed, 20 Apr 2022 00:00:00 +0000https://www.depends-on-the-definition.com/test-error-messages-with-pytest/In this short article, you will learn, how and when to test the error message of an exception with pytest.https://www.depends-on-the-definition.com/unsupervised-text-embeddings-with-transformers/Mon, 24 May 2021 00:00:00 +0000https://www.depends-on-the-definition.com/unsupervised-text-embeddings-with-transformers/In this article, we look at SimCSE, a simple contrastive sentence embedding framework, which can be used to produce superior sentence embeddings, from either unlabeled or labeled data. The idea behind the unsupervised SimCSE is to simply predicts the input sentence itself, with only dropout used as noise.https://www.depends-on-the-definition.com/missing-guide-on-data-preparation-for-language-modeling/Fri, 25 Sep 2020 00:00:00 +0000https://www.depends-on-the-definition.com/missing-guide-on-data-preparation-for-language-modeling/Language models gained popularity in NLP in the recent years. Sometimes you might have enough data and want to train a language model like BERT or RoBERTa from scratch. While there are many tutorials about tokenization and on how to train the model, there is not much information about how to load the data into the model. This guide aims to close this gap.https://www.depends-on-the-definition.com/data-augmentation-with-transformers/Sun, 23 Aug 2020 00:00:00 +0000https://www.depends-on-the-definition.com/data-augmentation-with-transformers/In this article we sample from pre-trained transformers to augment small, labeled text datasets for named entity recognition.https://www.depends-on-the-definition.com/how-to-approach-nlp/Tue, 07 Jul 2020 00:00:00 +0000https://www.depends-on-the-definition.com/how-to-approach-nlp/This time, I’m going to talk about how to approach general NLP problems. But we’re not going to look at the standard tips which are tosed around on the internet, for example on platforms like kaggle.https://www.depends-on-the-definition.com/data-validation-with-topic-models/Wed, 03 Jun 2020 00:00:00 +0000https://www.depends-on-the-definition.com/data-validation-with-topic-models/In a recent article, we saw how to implement a basic validation pipeline for text data. Once a machine learning model has been deployed its behavior must be monitored. The predictive performance is expected to degrade over time as the environment changes.https://www.depends-on-the-definition.com/lda-from-scratch/Wed, 20 May 2020 00:00:00 +0000https://www.depends-on-the-definition.com/lda-from-scratch/Today, I’m going to talk about topic models in NLP. Specifically we will see how the Latent Dirichlet Allocation model works and we will implement it from scratch in numpy. What is a topic model? Assume we are given a large collections of documents.https://www.depends-on-the-definition.com/data-validation-for-nlp/Thu, 30 Jan 2020 00:00:00 +0000https://www.depends-on-the-definition.com/data-validation-for-nlp/An important part of machine learning applications, is making sure that there is no data degeneration while a model is in production. Sometimes downstream data processing changes and machine learning models are very prone to silent failure due to this.https://www.depends-on-the-definition.com/confident-learning-for-nlp/Tue, 21 Jan 2020 00:00:00 +0000https://www.depends-on-the-definition.com/confident-learning-for-nlp/In every machine learning project, the training data is the most valuable part of your system. In many real-world machine learning projects the largest gains in performance come from improving training data quality. Training data is often hard to aquire and since the data can be large, quality can be hard to check.https://www.depends-on-the-definition.com/how-explainable-ai-fails-and-what-to-do-about-it/Sat, 28 Dec 2019 00:00:00 +0000https://www.depends-on-the-definition.com/how-explainable-ai-fails-and-what-to-do-about-it/This article heavily relys on "Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead" by Cynthia Rudin and finally on some of my personal experiences. I will mainly focus on technical issues and leave out most of the governance and ethics related issues that derive from these.https://www.depends-on-the-definition.com/how-the-lime-algorithm-fails/Tue, 10 Dec 2019 00:00:00 +0000https://www.depends-on-the-definition.com/how-the-lime-algorithm-fails/You maybe know the LIME algorithm from some of my earlier blog posts. It can be quite useful to “debug” data sets and understand machine learning models better. But LIME is fooled very easily.https://www.depends-on-the-definition.com/cluster-discovery/Sun, 24 Nov 2019 00:00:00 +0000https://www.depends-on-the-definition.com/cluster-discovery/If you are dealing with a large collections of documents, you will often find yourself in the situation where you are looking for some structure and understanding what is contained in the documents. Here I’ll show you a convenient method for discovering and understanding clusters of text documents.https://www.depends-on-the-definition.com/pycon-pydata-berlin-2019/Tue, 15 Oct 2019 00:00:00 +0000https://www.depends-on-the-definition.com/pycon-pydata-berlin-2019/Last weekendPyCon DE and PyData Berlin joined in Berlin for a great conference event that I was lucky to attend. The speaker line-up was great and often it was hard to choose which talk or tutorial to attend.https://www.depends-on-the-definition.com/model-uncertainty-in-deep-learning-with-monte-carlo-dropout/Mon, 05 Aug 2019 00:00:00 +0000https://www.depends-on-the-definition.com/model-uncertainty-in-deep-learning-with-monte-carlo-dropout/Deep learning models have shown amazing performance in a lot of fields such as autonomous driving, manufacturing, and medicine, to name a few. However, these are fields in which representing model uncertainty is of crucial importance.https://www.depends-on-the-definition.com/interpretable-named-entity-recognition/Sat, 08 Jun 2019 00:00:00 +0000https://www.depends-on-the-definition.com/interpretable-named-entity-recognition/In the previous posts, we saw how to build strong and versatile named entity recognition systems and how to properly evaluate them. But often you want to understand your model beyond the metrics. So in this tutorial I will show you how you can build an explainable and interpretable NER system with keras and the LIME algorithm.https://www.depends-on-the-definition.com/introduction-to-embeddings-with-neural-networks/Sun, 14 Apr 2019 00:00:00 +0000https://www.depends-on-the-definition.com/introduction-to-embeddings-with-neural-networks/Since a lot of people recently asked me how neural networks learn the embeddings for categorical variables, for example words, I’m going to write about it today. You all might have heard about methods like word2vec for creating dense vector representation of words in an unsupervised way.https://www.depends-on-the-definition.com/introduction-n-gram-language-models/Sun, 07 Apr 2019 00:00:00 +0000https://www.depends-on-the-definition.com/introduction-n-gram-language-models/You might have heard, that neural language models power a lot of the recent advances in natural language processing. Namely large models like Bert and GPT-2. But there is a fairly old approach to language modeling that is quite successful in a way.https://www.depends-on-the-definition.com/identify-ingredients-with-neural-networks/Fri, 08 Mar 2019 00:00:00 +0000https://www.depends-on-the-definition.com/identify-ingredients-with-neural-networks/Today we want to build a model, that can identify ingredients in cooking recipes. I use the “German Recipes Dataset”, I recently published on kaggle. We have more than 12000 German recipes and their ingredients list.https://www.depends-on-the-definition.com/how-to-magnitude-with-keras/Tue, 12 Feb 2019 00:00:00 +0000https://www.depends-on-the-definition.com/how-to-magnitude-with-keras/This time we have a look into the magnitude library, a feature-packed Python package and vector storage file format for utilizing vector embeddings in machine learning models in a fast, efficient, and simple manner developed by Plasticity.https://www.depends-on-the-definition.com/text-analysis-with-named-entities/Wed, 26 Dec 2018 00:00:00 +0000https://www.depends-on-the-definition.com/text-analysis-with-named-entities/This is the second post of my series about understanding text datasets. If you read my blog regularly, you probably noticed quite some posts about named entity recognition. In this posts, we focused on finding the named entities and explored different techniques to do this.https://www.depends-on-the-definition.com/named-entity-recognition-with-bert/Mon, 10 Dec 2018 00:00:00 +0000https://www.depends-on-the-definition.com/named-entity-recognition-with-bert/In 2018 we saw the rise of pretraining and finetuning in natural language processing. Large neural networks have been trained on general tasks like language modeling and then fine-tuned for classification tasks. One of the latest milestones in this development is the release of BERT.https://www.depends-on-the-definition.com/understanding-text-data-with-topic-models/Sat, 24 Nov 2018 00:00:00 +0000https://www.depends-on-the-definition.com/understanding-text-data-with-topic-models/This is the first post of my series about understanding text datasets. A lot of the current NLP progress is made in predictive performance. But in practice, you often want and need to know, what is going on in your dataset.https://www.depends-on-the-definition.com/attention-lstm-relation-classification/Wed, 19 Sep 2018 00:00:00 +0000https://www.depends-on-the-definition.com/attention-lstm-relation-classification/Once named entities have been identified in a text, we then want to extract the relations that exist between them. As indicated earlier, we will typically be looking for relations between specified types of named entity.https://www.depends-on-the-definition.com/evaluate-sequence-models/Fri, 24 Aug 2018 00:00:00 +0000https://www.depends-on-the-definition.com/evaluate-sequence-models/An important part of every machine learning project is the proper evaluation of the performance of the system. In this post we will talk about evaluation of token-based sequence models. This is especially tricky because:https://www.depends-on-the-definition.com/test-time-augmentation-keras/Thu, 09 Aug 2018 00:00:00 +0000https://www.depends-on-the-definition.com/test-time-augmentation-keras/In the last post, I introduced the U-Net model for segmenting salt depots in seismic images. This time, we will see how to improve the model by data augmentation and especially test time augmentation (TTA). You will learn how to use data augmentation with segmentation masks and what test time augmentation is and how to use it in keras.https://www.depends-on-the-definition.com/unet-keras-segmenting-images/Thu, 26 Jul 2018 00:00:00 +0000https://www.depends-on-the-definition.com/unet-keras-segmenting-images/Today I’m going to write about a kaggle competition I started working on recently. In the TGS Salt Identification Challenge, you are asked to segment salt deposits beneath the Earth’s surface. So we are given a set of seismic images that are $101 \times 101$ pixels each and each pixel is classified as either salt or sediment.https://www.depends-on-the-definition.com/named-entity-recognition-with-residual-lstm-and-elmo/Sun, 01 Jul 2018 00:00:00 +0000https://www.depends-on-the-definition.com/named-entity-recognition-with-residual-lstm-and-elmo/This is the sixth post in my series about named entity recognition. This time I’m going to show you some cutting edge stuff. We will use a residual LSTM network together with ELMo embeddings, developed at Allen NLP. You will learn how to wrap a tensorflow hub pre-trained model to work with keras. The resulting model with give you state-of-the-art performance on the named entity recognition task.https://www.depends-on-the-definition.com/debugging-black-box-text-classifiers-with-lime/Sat, 02 Jun 2018 00:00:00 +0000https://www.depends-on-the-definition.com/debugging-black-box-text-classifiers-with-lime/Often in text classification, we use so called black-box classifiers. By black-box classifiers I mean a classification system where the internal workings are completely hidden from you. A famous example are deep neural nets, in text classification often recurrent or convolutional neural nets.https://www.depends-on-the-definition.com/keras-and-eli5/Sat, 02 Jun 2018 00:00:00 +0000https://www.depends-on-the-definition.com/keras-and-eli5/In this post, I’m going to show you how you can use a neural network from keras with the LIME algorithm implemented in the eli5 TextExplainer class. For this we will write a scikit-learn compatible wrapper for a keras bidirectional LSTM model. The wrapper will also handle the tokenization and the storage of the vocabulary.https://www.depends-on-the-definition.com/pydata-amsterdam-2018/Tue, 08 May 2018 00:00:00 +0000https://www.depends-on-the-definition.com/pydata-amsterdam-2018/Last weekend I participated at the PyData Amsterdam 2018 Conference in, you guess it, in Amsterdam. It has been a great conference and I meet a lot of great people and had a very good time in Amsterdam.https://www.depends-on-the-definition.com/lstm-with-char-embeddings-for-ner/Sun, 15 Apr 2018 00:00:00 +0000https://www.depends-on-the-definition.com/lstm-with-char-embeddings-for-ner/This is the fifth post in my series about named entity recognition. If you haven’t seen the last four, have a look now. The last time we used a CRF-LSTM to model the sequence structure of our sentences.https://www.depends-on-the-definition.com/guide-to-word-vectors-with-gensim-and-keras/Fri, 16 Mar 2018 00:00:00 +0000https://www.depends-on-the-definition.com/guide-to-word-vectors-with-gensim-and-keras/Word vectors Today, I tell you what word vectors are, how you create them in python and finally how you can use them with neural networks in keras. For a long time, NLP methods use a vectorspace model to represent words.https://www.depends-on-the-definition.com/transfer-learning-for-dog-breed-identification/Sat, 03 Feb 2018 00:00:00 +0000https://www.depends-on-the-definition.com/transfer-learning-for-dog-breed-identification/This time I will show you how to build a simple “AI” product with transfer learning. We will build a “dog breed identification chat bot”. In this first post, I will show how to build a good model using keras, augmentation, pre-trained models for transfer learning and fine-tuning.https://www.depends-on-the-definition.com/detecting-network-attacks-with-isolation-forests/Sat, 27 Jan 2018 00:00:00 +0000https://www.depends-on-the-definition.com/detecting-network-attacks-with-isolation-forests/In this post, I will show you how to use the isolation forest algorithm to detect attacks to computer networks in python. The term isolation means separating an instance from the rest of the instances. Since anomalies are ‘few and different’ and therefore they are more susceptible to isolation.https://www.depends-on-the-definition.com/classify-toxic-comments-on-wikipedia/Sat, 23 Dec 2017 00:00:00 +0000https://www.depends-on-the-definition.com/classify-toxic-comments-on-wikipedia/This time we’re going to discuss a current machine learning completion on kaggle. In this competition, you’re challenged to build a multi-headed model that’s capable of detecting different types of of toxicity like threats, obscenity, insults, and identity-based hate.https://www.depends-on-the-definition.com/sequence-tagging-lstm-crf/Mon, 27 Nov 2017 00:00:00 +0000https://www.depends-on-the-definition.com/sequence-tagging-lstm-crf/This is the fourth post in my series about named entity recognition. If you haven’t seen the last three, have a look now. The last time we used a recurrent neural network to model the sequence structure of our sentences.https://www.depends-on-the-definition.com/guide-sequence-tagging-neural-networks-python/Sun, 22 Oct 2017 00:00:00 +0000https://www.depends-on-the-definition.com/guide-sequence-tagging-neural-networks-python/This is the third post in my series about named entity recognition. If you haven’t seen the last two, have a look now. The last time we used a conditional random field to model the sequence structure of our sentences.https://www.depends-on-the-definition.com/named-entity-recognition-conditional-random-fields-python/Sun, 10 Sep 2017 00:00:00 +0000https://www.depends-on-the-definition.com/named-entity-recognition-conditional-random-fields-python/This is the second post in my series about named entity recognition. If you haven’t seen the first one, have a look now. Last time we started by memorizing entities for words and then used a simple classification model to improve the results a bit.https://www.depends-on-the-definition.com/introduction-named-entity-recognition-python/Sat, 26 Aug 2017 00:00:00 +0000https://www.depends-on-the-definition.com/introduction-named-entity-recognition-python/In this post, I will introduce you to something called Named Entity Recognition (NER). NER is a part of natural language processing (NLP) and information retrieval (IR). The task in NER is to find the entity-type of words.https://www.depends-on-the-definition.com/classifying-genres-of-movies-by-looking-at-the-poster-a-neural-approach/Sat, 12 Aug 2017 00:00:00 +0000https://www.depends-on-the-definition.com/classifying-genres-of-movies-by-looking-at-the-poster-a-neural-approach/In this article, we will apply the concept of multi-label multi-class classification with neural networks from the last post, to classify movie posters by genre. First we import the usual suspects in python. import numpy as np import pandas as pd import glob import scipy.https://www.depends-on-the-definition.com/guide-to-multi-label-classification-with-neural-networks/Fri, 11 Aug 2017 00:00:00 +0000https://www.depends-on-the-definition.com/guide-to-multi-label-classification-with-neural-networks/Often in machine learning tasks, you have multiple possible labels for one sample that are not mutually exclusive. This is called a multi-class, multi-label classification problem. Obvious suspects are image classification and text classification, where a document can have multiple topics.https://www.depends-on-the-definition.com/efficient-aws-for-deep-learning/Tue, 01 Aug 2017 00:00:00 +0000https://www.depends-on-the-definition.com/efficient-aws-for-deep-learning/When running experiments with deep neural nets you want to use appropriate hardware. Most of the time I work on a thinkpad laptop with no GPU. This makes experimenting painfully slow. A convenient way is to use an AWS instance, for example the p2.https://www.depends-on-the-definition.com/getting-started-with-multivariate-adaptive-regression-spline/Sun, 30 Jul 2017 00:00:00 +0000https://www.depends-on-the-definition.com/getting-started-with-multivariate-adaptive-regression-spline/In this post we will introduce multivariate adaptive regression splines model (MARS) using python. This is a regression model that can be seen as a non-parametric extension of the standard linear model. The multivariate adaptive regression splines model MARS builds a model of the fromhttps://www.depends-on-the-definition.com/about/Mon, 01 Jan 0001 00:00:00 +0000https://www.depends-on-the-definition.com/about/the author Hi, my name is Tobias. I’m a trained mathematician that now works as a data scientist and machine learning engineer. In 2018 I achieved the status of a kaggle master and still spend quite some time there.https://www.depends-on-the-definition.com/imprint/Mon, 01 Jan 0001 00:00:00 +0000https://www.depends-on-the-definition.com/imprint/Contact E-Mail: info@depends-on-the-definition.com Accountability for content The contents of our pages have been created with the utmost care. However, we cannot guarantee the contents’ accuracy, completeness or topicality. According to statutory provisions, we are furthermore responsible for our own content on these web pages.https://www.depends-on-the-definition.com/privacy/Mon, 01 Jan 0001 00:00:00 +0000https://www.depends-on-the-definition.com/privacy/It is Depends-on-the-definition’s policy to respect your privacy regarding any information we may collect while operating our website. This Privacy Policy applies to https://depends-on-the-definition.com (hereinafter, “us”, “we”, or “https://depends-on-the-definition.com”). We respect your privacy and are committed to protecting personally identifiable information you may provide us through the Website.https://www.depends-on-the-definition.com/work-with-me/Mon, 01 Jan 0001 00:00:00 +0000https://www.depends-on-the-definition.com/work-with-me/I’m a freelancer I have a passion to support companies to solve their business challenges by leveraging their data and applying advanced machine learning. Services offered Hands on teaching/consulting - Learn how to solve your problems with code, the client chooses what is covered/taught.