文本数据分析实战【数据清洗、统计分析、可视化展示、情感分析】

      文本数据分析里面情感分析的应用十分广泛，本质上来说就是一个分类任务，在我之前的文章里面对有对中文数据的情感分析相关的工作，对于英文数据的分析还没有实践过，这里就想基于英文数据集来做一点分析性的工作，首先来看一下数据样例，如下所示：
      下面是100条数据样例：

ID,Product_ID,Age,Review_Title,Review_Text,Rating,Recommended,Positive_Feedback_Count,Division,Department,Type
12906,984,57,Unique,"Just received and sorry to return. i am 5'9"" i wear a size 8 i ordered the m/l . it way really big and to wide",4,1,1,General Petite,Jackets,Jackets
22440,872,60,Simple beauty,This is a tts basic tee with the added flare of the ruffled back peplum and the elegance of the beaded front neckline. the beading is lovely and mine was intact. i bought my normal small based on reviews and the fit was perfect. (34-b) the beige color is a classic neutral and can be worn year round by just adding a jacket or sweater. love it!,5,1,3,General Petite,Tops,Knits
13965,862,35,Love this top!,I am very happy with this shirt. i don't like skin tight clothes. i bought this top a bit loose but it does not look like i am wearing a sack. it is tunic length on me so i have worn it with jeans and boots. i love the way it flows while i walk. very flattering on. would definitely recommend!,5,1,0,General,Tops,Knits
14369,829,45,,"Really flattering cut, this shirt is perfect for fall. got so many compliments the first time i wore it!",5,1,0,General Petite,Tops,Blouses
505,833,56,,Love the fabric & lace on this top. like other reviews said the arm area is tight where the lace meets the fabric which makes the fit uncomfortable. sadly this top is going back. if you have very skinny upper arms this top will fit fine.,4,1,0,General,Tops,Blouses
19166,1121,25,"Beautiful design, poor execution","I knew i needed to buy this trench from the moment i saw it on my computer screen, but something prompted me to stop by my local retailer and try it on first. thank goodness for that.



at first glance, i knew i wouldn't be buying this. the construction leaves much to be desired. from the flimsy, single layer of unlined, unstructured fabric, to the thin strip of fabric serving as a belt, i could easily see this being sold at a cheap store for teenage girls where throwaway clothes are a dim",1,0,4,General,Jackets,Outerwear
12522,1059,25,Adorable on!!!,This is so cute with a shirt or tee under neath. the look is better with a shirt underneath. very adorable for many occassions. i ordered the petite sizing b/c i like my pants more cropped.,5,1,3,General,Bottoms,Pants
15427,1099,39,I agree,"I agree with all the positive reviews about the look, fit and comfort of this dress.  definitely a four season dress.  for winter i plan on wearing wool tights, wool coat and boots and keeping the waist untied.  for me both the medium petite & regular looked good just a bit tight in the shoulders, since that's how my body is built.  i'm keeping the regular hem version.  my stats for fit reference: 5'3"" 140# 36dd, meaty arms/shoulders/back, small hips, no waist/bum.",5,1,0,General Petite,Dresses,Dresses
1178,1204,39,Comfy and cute pants,I love these pants. i have them in navy and carbon. the navy color seems to run bigger than the carbon. they are very loose like a boyfriend pant in navy. they loosen up as the day goes on as well. super comfy and great for running errands and taking my little one to the park. i can look cute and feel comfy and i can give my yoga pants a break.,5,1,3,General Petite,Bottoms,Pants
16718,1037,54,These cords are great!,"These cords are beautiful in the wine color and fit very well! i am 5'1 and got the petite size and felt they were a little to short, i know they are cropped flare but they hit me at a weird place. hindsight, i probable should have ordered then one size up!",5,1,0,General Petite,Bottoms,Jeans
19003,1052,42,Boxy,Great color and quality but very boxy. even with the drawstring there is no shape going on here.,3,0,0,General Petite,Bottoms,Pants
4696,831,47,Stunningly beautiful,"I took a risk and bought this blouse and got a wonderful surprise. i bought my usual size small (34c, 31"" waist) and it was perfect. the details of this blouse are amazing. you cannot see the flecks of copper-gold leaves all over the fabric or the gold threads running down the seam in the front. it is really, really beautiful. the fabric overall is soft and flowy and the sleeves do not stand out off my shoulders in a weird way. there is a soft flow of ruffle down both sides of the shirt. the fro",5,1,3,General,Tops,Blouses
4282,1081,53,Neckline bothers me,"I like the fabric - very smooth and feels nice on the skin. i like the handkerchief hemline too.i didn't realize how much i would not like the neckline. it feels too tight and too high to me and i'm not sure i could wear this for long without pulling at the neck. also, the fit is loose, maybe a bit too loose for my taste. but it's the neck fit that makes me have to return this dress.",3,0,28,General,Dresses,Dresses
18168,1094,46,"Beautiful, flattering high-quality dress","This dress is a home run on all accounts. the embroidery is gorgeous, the fabric is high-quality, and the a-line cut is slimming. i found this dress to run true to size. i am 5'11"" and the length hits above the  knee--so it could be longer on shorter heights. i purchased the white dress initially and now have also ordered the navy.",5,1,3,General,Dresses,Dresses
23287,823,45,"Too much fabric, shoulders won't stay down","This is a gorgeous print, but i found it too fussy for my taste.  it is quite shapeless so i tried doing a half tuck. it looked cute for about 2 seconds, but then i had to worry about keeping the shoulders down.",2,0,0,General,Tops,Blouses
17222,1077,31,Very chic!,This dress looks very good on me! it's a cute way to appear dressy without the extra work to look the part.,5,1,0,General Petite,Dresses,Dresses
6980,719,60,"So pretty, but so sheer","The material and design of this beach dress is so pretty. i was hoping that this could be worn as both a dress and a swim suit cover-up, but it is only the latter. you can't see it from the photos (the model or mine), but it is very sheer.... it can really only be worn over a swimsuit. whatever you wear (or don't) will be completely visible. so if you are looking for just a lovely, unique beach dress, this is it! unfortunately for the price i need more versatility! i will consider it on sale!",4,1,32,Initmates,Intimate,Lounge
2161,872,30,Comfort and style to the max!,Love this shirt. i am 5'8 and have a slender torso and the small fits perfectly. i love that it's long so you can wear it as a single shirt or use it as an extra layer. i also have a grey druzy necklace to dress this puppy up. purchased it in black and this will be a new staple.,5,1,0,General Petite,Tops,Knits
3075,1009,36,Adorable all season skirt,"The gorgeous coral color of this skirt makes it a great all seasons skirt. the slightly heavier weight of skirt makes it extremely adorable for fall/winter. pair it with some tights, chunky knit and booties, good to go. very flattering one. has a slight a-line cut to it. i love love this one.",5,1,1,General,Bottoms,Skirts
20194,1004,31,Pretty but not perfect,"This is a thick jacquard fabric and the colors are stunning. this would be best for very thin petties or tall women. i'm 5'4"", 125#, 27/4, and it was just too much fabric so my waist looked thick. the length was great though. sad it didn't work out",3,1,5,General,Bottoms,Skirts
20656,949,49,Perfect jacket,"I love my new blazer! the material is very soft and comfortable, and very nice quality, but not too ""wooly"" to wear into spring. the cut is very flattering, and the patterns and fringe details are very eye-catching. i am 5'6"" and usually wear a size s in retailer and this small fit perfectly. they grey/cream color combo is gorgeous and looks very versatile, and can be dressed up or down.



the one alteration i suggest is moving the button; at the shoulder it gives a wide boxy shape. my tailor wil",5,1,1,General,Tops,Sweaters
4415,1078,41,Figure flattering and the material is so soft!,"I bought this dress in the navy stripe version and love it. the material is so soft it's like wearing pajamas, and the actual swing skirt is flattering (which surprised me when i tried it on). i wore it to work with burgundy heels and got a lot of compliments from my co-workers. i'm 5'9"", 130 lbs and the xs worked best for me.",5,1,0,General Petite,Dresses,Dresses
12157,1068,42,"Cute pants, but size down","I just received these in the mail, but had to send them back to the store because they are too big. i usually wear a 28 and sometimes even a 29 in most ag and mother jeans. the 28 in these were loose in the waist and overall baggier than shown in the picture. i ordered size 27, so hopefully these will fit just right since they are really cute. the beige color is just as pictured and almost a rust color which goes well with a dark grey top as shown.",5,1,5,General,Bottoms,Pants
17784,861,33,Love this shirt!,"I ordered this shirt in both white and coral. i absolutely love the design and coral color. the white was a bit see through, so i returned it. it would be cute with a cami under, but i just wanted something that did not require an extra layer. overall, it is a flattering fit and i love the detail on the back.",5,1,0,General,Tops,Knits
13724,1078,49,Pretty but odd fit,"The fabric is nice and could pass for silk even though it is viscose-rayon. the pattern is pretty and flattering. sadly, the fit was just off- big in the center but tight at the hips. if i size up the v-neck might be too wide. i'm disappointed that i'll have to return it. it probably would look great on a different body type. i would recommend it for apple body types but not for pear body types.",3,1,0,General,Dresses,Dresses
20217,1078,48,"Cute, but...","Wanted to love this dress, bit it wasn't to be. strange fit, and doesn't drape nicely, as in the photo. also, the blue color was much darker in person. wouldn't recommend.",2,0,0,General,Dresses,Dresses
20634,1084,54,Love love love!,"I love this dress! i was so excited when i saw this hanging in my local retailer, just one in my size (it must have been a return). i tried it on and fell in love with it! i am 34b bra size, 130 pounds, 5'9 tall and an xs fits me great! i didn't find it to be tight on top for me, as other reviewers have said, and the skirt is very flowy. i have worn this dress so many times since i bought it and have gotten compliments on it each time. i bought the green color-- its a beautiful print, and usually",4,1,0,General,Dresses,Dresses
18969,1084,46,Needs pockets! runs large,"Even though this is an online exclusive, i was able to try on a one off in the store. i have been looking for a cross front dress and the other option currently at retailer (the black and white striped top version) in a similar length) looked awkward. this dress looks great on but i needed the xs. i usually wear a 2. it was cute and i came online to see if the xs was available but it is sold out. the dress would be perfect if it had pockets!",4,1,1,General,Dresses,Dresses
3580,950,36,Swoon-worthy!,"Once this caught my eye, i couldn't let it go. i kept revisiting it...thinking it was too expensive, probably itchy,

too bulky, a problem if you wanted to eat something.

but the color looked yummy so i finally ordered it....

and i'm so pleased i did! 

the shape is actually easy to wear - it's not chopped. it drapes nicely over my hips and it isn't itchy.

i would not wash the dishes with this sweater on...but you could dress it up with velvet pants for a party or 

more casually with jeans.",5,1,1,General,Tops,Sweaters
14407,1078,38,Not worth the effort,"Generally, i find sizing at retailer to be very consistent, so i ordered this dress in the same size that i order all of my other retailer dresses. the bust was way to small. so i had to go and return this and have the next size up shipped to me. it's till way too small! in addition, the shipping was terrible - the dress was just dumped into a bag with no attempt to fold this, and the right strap was defective! it's so irritating that i have to return this again!",1,0,0,General Petite,Dresses,Dresses
9196,862,73,I live in these tanks!,"I'm a power-walker. i bought this tank in 3 colors and rotate through them. they're great for working out with a nice sports bra underneath. i also wear them with leggings. so cute! love the colors, the fit. yes, they're thin. i don't think they'll last for years and years, but with the sale price they are such a great deal.",5,1,0,General,Tops,Knits
16894,834,52,Great shape and flow,"This is a great piece, which has flow, grace, and charm. the blue is a beautiful color! i love it and received numerous complements after wearing it one time.",5,1,2,General,Tops,Blouses
23193,1087,49,Love love love this,"I love this sweater dress. i got the ivory color in a size small. i'm a size 6 & 5'5"". i feel in love with it while shopping with a friend, who kind of rolled her eyes at it. i went back later because i knew i loved it. since the ones on the sale rack were covered with dust and dirt at the hem. i ordered it on line in m & small and the small fit better. it is so soft, and the color is great.",5,1,0,General Petite,Dresses,Dresses
9825,1080,33,Short,I really loved this dress. it ran small but it was also shorter than i imagined which was only worsened because i'm curvier on top. would be great for slender figures.,3,0,5,General,Dresses,Dresses
11853,1059,30,The only jumpsuit i own,"I am not really a fan of jumpsuits/rompers/etc. but i love this jumpsuit. very good quality and design. fits well, but i thought the 4 was pretty large for a 4. i mean, i probably should have gotten a 2 (which was sold out), but i don't usually have to get a 4 tailored. i did in this case, but it was still worth it.",5,1,0,General,Bottoms,Pants
12855,1059,61,Great pant,"Wow! great find and a true fit. i totally dislike pant shopping and these are a gem.

slimming, flattering and so comfortable, the style is great. great basic summer pant!",5,1,0,General,Bottoms,Pants
11964,829,43,Didn't work!,"I saw this very shirt online & thought it was so pretty. in person, the shirt seems flimsy. the buttons look as though they'd fall off. i also didn't like the cut of the shirt. it seemed to be tighter on top & looser on the bottom. had to return. i couldn't justify the cost with the quality.",1,0,2,General Petite,Tops,Blouses
20691,833,36,Still not sure about this,"After seeing the picture online, i was coveting this top for a while and then retailer days happened! i immediately ordered it, but i'm still not sure if i'm going to keep it. i ordered a 2p (0p was sold out) and i was assuming it was going to be a little big, but i like flowy. the sleeves and chest are perfect but the top is cut so generously in the back (and it's a bit longer in the back as well) i feel like it looks like i'm wearing a wedding dress. i love it from the front. i'm going to try to",3,0,1,General,Tops,Blouses
22113,169,66,,"Love these leggings!!! soft, cozy and yummy feeling! can't wait for the weather to get cooler! these fleece-lined leggings will be a new staple this winter!!!",5,1,3,Initmates,Intimate,Legwear
22992,1078,35,"Take car with it, but it's gorgeous","I love this dress. i agree with the other reviewers that the rhinestone linkages do very easily get caught on the lace part of the dress and i could see it snagging very quickly. that said, i am going to keep it and hope for the best because it is absolutely gorgeous. it is not actually a true white as it looks pictured. instead we, it is more of a nude color, but it adds to the flattering nature of the dress and makes it look sheer. the embroidery is beautiful. my only complaint is that it will",4,1,8,General Petite,Dresses,Dresses
18674,835,56,Gorgeous top!,I'm obsessed with this top! i got it for christmas and wore it to dinner with black pants and flats and got lots of compliments. the fit is cute and fun and the detail is about even prettier in person.,5,1,0,General,Tops,Blouses
17529,1083,45,Beautiful design,"I bought this dress online after a long search for the perfect dress to wear to my daughter's fall wedding. i did not want something full length or too formal. after modeling several dresses for my boys this one got the thumbs up. i do love the design and that it is different enough from all the other mother-of-bride dresses out there. i will wear this with black nylons and wedges (since this wedding is in a meadow). i'm completely satisfied! for reference i am 45 years old, 5'10"", 160 lbs with",5,1,3,General Petite,Dresses,Dresses
17372,1068,46,Favorite linen pants,I love these pants so much i had to write a review! purchased the carbon color recently in regular length - the length stated is pretty true and there was minimal shrinkage for me. size was also tts (typically wear size 2 and got 26). i've been wearing them often! they look great right out of the dryer and drape nicely. i've been purchasing the pilcro linen trousers for years but these are my favorites this season --- i also think the quality is much better. i will be getting them in more colors,5,1,9,General,Bottoms,Pants
21636,1077,34,,"I wasn't sure about this dress, and decided to give it a shot based on reviews, could not be happier. before trying it on i was worried about the length, and always worry about dreaded button gap at the chest, but none of that is an issue.",5,1,0,General,Dresses,Dresses
13782,1078,45,Definitely size down,"Love everything about this dress except - it runs huge.  i am busty and usually wear a 12 in dresses, but the large was completely shapeless on me.  will try medium and hope it is better bc otherwise i love the dress!!",4,1,0,General,Dresses,Dresses
8501,861,42,Summer's must have tee!,"I love these t.la tees, and i buy them often. i find them to be very versatile and super soft. i have several colors and they hold up well if you wash and hang dry. i am a small in most retailer tops and these are no exception. they wear well under jean jackets or even a long-sleeve button-down shirt.",5,1,1,General,Tops,Knits
23472,855,32,Perfectly drapey,"I saw the shirt on the retailer website and new i had to have it. when it arrived, it was perfectly draped, light and airy, and incredibly soft and comfortable.  it's perfect to throw on with yoga pants for errands or dress up a bit a casual night out with friends. would buy this again in several different colors if they were offered. 9/10.",5,1,0,General,Tops,Knits
8857,1088,30,Huge,"This dress is huge. i am normally a l-xl regular in retailer brands. i love a loose, flowy dress. after trying one on in the store (just a random returned petite xl) i ordered the petite medium. and it was still huge and not flattering. so sad. i wanted to love this- loved the material and colors, but way off.",1,0,4,General,Dresses,Dresses
11389,1081,54,Cute and easy,"I was excited to find this on sale because i just love baily 44, their designs are always simple, beautiful, and comfortable, and this one, just like the other 2 baily 44 dresses i own is perfect. i am 5'9, 125 pounds, and 34b bra size and got my usual size, xs. unlike my other baily 44 dresses, this one has a more relaxed fit up top and then has a form fitting layer that shows off your curves on the bottom. i personally don't think it's too short, but then again plan to wear tights with this pi",4,1,1,General Petite,Dresses,Dresses
20013,853,34,"Yup, no ruffle.","Sooo this adorable blouse is indeed different than pictured. no ruffle, such a bummer, sleeves are much longer, and yes it's teared in the front and back. also the button opening on the back is much longer/larger, in my opinion. i'll most likely keep it, but consider the mentioned if buying.",3,1,1,General,Tops,Blouses
12881,1078,32,Had to return,"I loved this dress and it fit perfectly. i did not have any issue with the camisole being tight, and i am really curvy. i did however have a problem trying to take it off. it has no zipper and in order for me to get it over my head i had to stretch my shoulders to the point where i ripped the back. i went to the store and tried on the larger size and ripped that one too. i tried to take it off more delicately but i couldn't. maybe it was me, but this dress is too much hassle. if i went any highe",2,0,2,General Petite,Dresses,Dresses
7973,862,59,Runs quite large,"The photo of the mila turtleneck where model has her hands to her sides, and just below her hair, there is a bit of extra fabric which falls to the bottom of the sweater in a bit of a pleat- well, it's not representative of the abundance of fabric in the xl. i usually wear an l or xl well with retailer clothing. the excess added the appearance of excessive pounds from breasts to bottom of sweater. bummer. i'll be returning it.",2,0,3,General,Tops,Knits
14180,862,33,Unique staple,"Fits true to size, soft material and unique look. the sleeves and overall cut of this shirt are flattering. i was worried it might look boxy or baggy but it didn't at all. the dark gray is a gorgeous color and can be dressed up or down. definitely going to be a staple in my closet.",5,1,0,General,Tops,Knits
15812,974,37,The pictures don't do it justice!,"This is one you just need to see in person! stunning, detailed embroidery everywhere, gracefully oversized, very bohemian, light, airy and dreamy. this kimono was mine the second i saw it! white on white is classic...and this cream embroidery is beautiful. the kimono is very oversized. i'm normally a l or xl and this was plenty roomy. it's also quite long. nearly came down to my knees and i'm around 5'8. if there is an option for petite i'd recommend it to shorter gals! can't wait to pair this w",5,1,24,General,Jackets,Jackets
2606,1110,44,Hourglass figure gals rejoice,"This dress is very feminine and flattering on, especially for hourglass figure gals. i usually wear size 4 regular in this brand but i sized up to 6 regular (i'm 34b, 26 waist, 36 hips). when it's very fitted on top, i like my dress to be a bit longer. the bottom hem falls about 2 inches longer on me than shown on the model (i'm 5'2""). it feels tight when first zipped because the dress is cut to fit the curves of a woman, but it loosens up after a few minutes of wear. the design of this dress is",5,1,2,General Petite,Dresses,Dresses
2181,872,42,,I need one in every color. so super soft.,5,1,1,General,Tops,Knits
2509,1081,29,Great basic,This dress fits great. i love that it's fitted at the top but drapery and swingy at the bottom.  it has a thicker t-shirt type fabric.  i will definitely style this with a variety of necklaces or a scarf and cardigan for cooler weather.  it's much cuter in person.  for reference i'm 5'4 105 and got xs.,5,1,2,General Petite,Dresses,Dresses
8572,873,33,Cute peplum tank without being frumpy,"I bought this tank on a whim without trying it on in the store, in black. the detail on the front is cute and the peplum is really cute without being frumpy at all. i am usually a small, ocassionally a medium in retailer shirts and i bought this in small. it fits but i feel like i could have gone up to a medium, it's just a little tight across the chest, but i do have more broad shoulders than most so maybe it's just because of that. the fabric is nice and stretchy, so i might be able to get away",5,1,14,General,Tops,Knits
20954,469,54,Two for one,"This is a high-quality swimsuit that can be worn inside out. easy to wear colors, fantastic pattern placement. the other side is an intense blue. i would suggest going by the size chart on the mara hoffman site. 

.

 i think this should look great on athletic figures. the back is low-cut and the bottom has medium coverage, not as skimpy as brazilian cut bottoms. not ideal for hour-glass figures as there is no bust support, and you will show quite a bit from the side (it is cut in at the front w",5,1,1,Initmates,Intimate,Swim
888,1026,26,Buy these now.,"These cords are seriously fantastic. worth the price indeed. they are very comfy, have a lot of stretch but do not loose shape! the fun yet muted color design allows an average outfit to have a fun pop. they do run big, i got a 26 and i'm usually a 27. they are also very long! nothing a little trip to tailors can't fix. overall, highly recommended.",5,1,0,General Petite,Bottoms,Jeans
3841,831,35,Pretty,"The colors on this are beautiful! i have wider hips and thighs and this blouse just looks too boxy on me. i ordered a size down and from that aspect it fit. however, it is too flowy and boxy for my body type. i considered keeping it for layering but even on sale, it is too expensive for this purpose. i'm so bummed because the colors are so pretty!",4,1,0,General,Tops,Blouses
13251,167,48,,"These are comfy and cool. i got the blue, which is more royal than navy. in opinion, it goes well with almost anything! i can wear all the neutral colors, in addition to orange, red, yellow and patterns. they are flattering and like other reviewers, i went up a size to 28 in the blue. perfect length, not too short. got the same version last year in white, 27. 5'4"", 128lbs",5,1,0,General,Bottoms,Shorts
4316,1078,46,Extremely awkward,"I bought this online so didn't realize that the fabric at the bottom is very, very stiff. i was expecting something silky, or at least on the softer and more moveable side. the best description i can think of is an overly starched mens dress shirt. that being the case, it kind of juts out in weird places. the sweater part is very nice, but it is somewhat closer fitting than what it seems in the picture. the fabric bottom is too ample for the slimmer design of the sweater which just adds to the o",1,0,9,General,Dresses,Dresses
31,1060,46,Cuter in oerson!,"I never would have given these pants a second look online, in person they are much cuter! the stripes are brighter and the fit more flattering. the crop has a cute flare which is right on trend. this brand has always run small for me, i am 5'8 about 140lbs and carry some chubbiness in the belly. i paired it with a collarless loose navy blazer",5,1,7,General Petite,Bottoms,Pants
11035,1093,53,Gorgeous but not flattering,"Very pretty dress and nicely made (the embroidery is quite lovely and double lining, as mentioned by the other reviewers, is awesome), but not a good shape for me and my hourglass figure. like a lot of swing dresses, it has no structure, and it's not a good look on someone a little busty, as i am. in all honesty, i thought the sack-like shape of the dress made me look 20 lbs heavier than i am. i am a medium, size 6-8, and took the 8 based on the other reviews. i probably could have sized down to",3,1,0,General,Dresses,Dresses
1559,1077,29,Cute & unique,This is such a unique and fun dress. i'm so glad i found it and it fits perfectly!,5,1,0,General Petite,Dresses,Dresses
6008,873,47,Adorable,"This top is adorable ,i finally broke down and bought it,so happy i did fits good,not too long sits at hip were it should (very flattering) i went with the reviews and sized up,i am a petite sx normally in retailer,but went with what the reviews said and got a reg small glad i did!it is a bit snug(not tight) in the chest,but has stretch to it , i am 5'1 108 33d (if you are bigger in the chest i would recommend a medium,) i got the purple motif (print) no see thru issues,the material is a comfortab",5,1,0,General,Tops,Knits
20290,1035,52,,,5,1,0,General Petite,Bottoms,Jeans
5293,867,70,Comfort & style in one great top!,"The design is loose and forgiving, and it is very comfortable. the fabric is soft and light weight, and i love how the sleeves are full cut so even those without stick arms can fit in them! it drapes beautifully and is quite lovely on. you can easily dress it up with jewelry or wear it over jeans for a great go-to top on the weekend. i also bought the purple version - it says it's mauve in the description, but it's really more of a pale lavender color. still very pretty, but i wish it was mauve!",5,1,2,General Petite,Tops,Knits
15616,949,44,Go to top,This is a great transition piece. wear with jeans or dress it up with a cute skirt.,4,1,0,General Petite,Tops,Sweaters
19797,1096,43,Gorgeous!,Oh i simply can't say enough about this dress... it's just so pretty! i've sometimes had trouble with beaded pieces from retailer but not this dress. it's a very flattering piece and the material is comfortable and soft. really flattering. you can dress it up or down as well - i'm glad i'll be able to wear this into fall with a leather jacket.,5,1,0,General,Dresses,Dresses
516,1081,53,Well made,This dress looks great on me. it gives a slender appearance which hides a lot and it's easy and comfortable to wear.,5,1,1,General Petite,Dresses,Dresses
16109,862,36,Nice but big,This runs a little big and falls off the shoulders a bit. i didn't like it as much as i had hoped.,3,1,0,General,Tops,Knits

     将原始的数据集划分为训练数据集和测试数据集，接下来简单看下数据样本情况：

      接下来对数据集nan值进行统计，如下所示：

train.isna().sum().sort_values(ascending=False)

test.isna().sum().sort_values(ascending=False)

      接下来看下Age字段的分布情况：

sns.distplot(train['Age'],fit=norm, bins=80)
(mu, sigma) = norm.fit(train['Age'])
plt.legend(['Normal dist. ($\mu=${:.2f} and $\sigma=${:.2f})'.format(mu, sigma)], loc='best')
plt.ylabel('Frequency')
plt.title('Distribution of Age agaist Normal')
plt.show()

     接下来看下我们标签类别字段Recomment的分布情况：

plt.hist(train['Recommended'],facecolor=(0.3,0.1,0.4,0.6),alpha=0.5,edgecolor='black')
plt.show()

     可以看到：两类数据的分布比例比较失衡。

     后面还可以做一些其他字段数据的分析与可视化，这里就不再具体说明了，如下所示：

     当然，好玩的可视化分析还可以有很多，这里就不再一一列举了。

      完成上面的工作后，下面就可以构建文本分类模型了，在构建模型之前，首先需要做的就是文本数据的向量化处理工作，这里有多种向量化方式可供选择，最简单的就是sklearn自带的tfidf向量化手段了，下面是简单的实现：

from sklearn.feature_extraction.text import TfidfVectorizer

corpus = list(trainx['Review_Text'])
tfidf_vectorizer = TfidfVectorizer(max_features=3400,analyzer='word',stop_words="english",ngram_range=(1,3),max_df = 0.9,min_df =3)
ti_trainx = tfidf_vectorizer.fit_transform(corpus).toarray()
ti_testx = tfidf_vectorizer.transform(list(testx['Review_Text'])).toarray()

print(ti_trainx.shape)
print(ti_testx.shape)



ti_trainx1 = pd.DataFrame(ti_trainx, columns = tfidf_vectorizer.get_feature_names(),index=merged_trainx.index)
print(ti_trainx1.shape)
x = pd.concat([ti_trainx1,merged_trainx],axis =1,ignore_index=False,verify_integrity=True)
print(x.shape)
y = train['Recommended']



ti_testx1 = pd.DataFrame(ti_testx, columns = tfidf_vectorizer.get_feature_names(),index=merged_testx.index)
print(ti_testx1.shape)
ti_xtest = pd.concat([ti_testx1,merged_testx],axis =1,ignore_index=False,verify_integrity=True)
print(ti_xtest.shape)

      也可以使用效果比较好的word2vec方法来实现文本数据的向量化处理，实现如下所示：

def word2vecModel(res_list,model_path='w2v.model'):
    '''
    词向量化模型
    '''
    data_list=[one_doc[0].strip().split(' ') for one_doc in res_list]
    label_list=[one_doc[-1].strip() for one_doc in res_list]
    print('data_list_length: ', len(data_list))
    print('label_list_length: ', len(label_list))
    model=word2vec.Word2Vec(data_list,sg=1,size=100,window=5,min_count=1,
                            negative=3,sample=0.001, hs=1,workers=4)
    model.save(model_path)
    return data_list,label_list,model

      输入词矩阵数据，输出训练好的词向量模型，之后可以用于生成每个词的向量表示。

      模型方面，我们直接基于sklearn来构建多种不同的模型，这里选用对比分析的模型主要包括：随机森林模型RF、支持向量机模型SVM、逻辑回归模型LR，具体实现如下所示：

model=LinearSVC()
dict1=cal_model_all_score(model,x,y,3)
print('dict1: ', dict1)
model=RandomForestClassifier(n_estimators=100)
dict2=cal_model_all_score(model,x,y,3)
print('dict2: ', dict2)
model=LogisticRegression(solver='liblinear')
dict3=cal_model_all_score(model,x,y,3)
print('dict3: ', dict3)
comparePloter(dict1,dict2,dict3,save_path='evaluation33.png')

     结果如下所示：

支持向量机SVM {'recall': 0.9605433137710847, 'F_value': 0.9332959221897774, 'precision': 0.9075517242383803, 'accuracy': 0.8868578809226783})
随机森林RF {'recall': 1.0, 'F_value': 0.9983786475647152, 'precision': 0.9967625441862978, 'accuracy': 0.9973235288175414})
逻辑回归LR {'recall': 0.9637913852286569, 'F_value': 0.9314406037171584, 'precision': 0.9011910746159048, 'accuracy': 0.8830864914770804})

      为了更加直观地看出来不同模型不同指标的情况，下面对上述结果进行可视化处理，如下所示：

     从上面四个指标综合来看，RF模型的综合性能要优于其他两种模型。这也证实了集成学习模型的强大。