2 years ago

#59031

test-img

Nitish Pandey

Problem while creating the NLP model using tensorflow?

I have created a spam classifier model using tensorflow.. ( tokenization , embedding layer and lstm ).

model=Sequential()
model.add(Embedding(voc_size,32,input_length=sent_len))
model.add(LSTM(units=128))
model.add(Dropout(0.5))
model.add(BatchNormalization())
model.add(Dense(units=1,activation='sigmoid'))

I have trained the model with my data and got an accuracy of about 97-98%.

I processed my data using the below function.

def process(data):
    data=re.sub('[^a-zA-Z]'," ",data)
    data=data.lower()
    data=data.split()
    data=[lemmatizer.lemmatize(word) for word in data if word not in set(stopword)]
    data=" ".join(data)
    return data

I have predicted my new custom input as :

def predict(data):
    data=[process(sent) for sent in data]
    data=tokenizer.texts_to_sequences(data)
    data=pad_sequences(data,maxlen=sent_len,padding='pre')
    pred=(mymodel.predict(data)>0.5).astype('int32').tolist()
    return pred

data=input()
predict(data)

But I am receiving the msg as non-spam for every msg I input..no matter what I enter.Look here

I don't know what is happening. I am not able to deploy such a model.

python

tensorflow

nlp

tokenize

word-embedding

0 Answers

Your Answer

Accepted video resources